MIT AI: OpenAI Meta-Learning and Self-Play (Ilya Sutskever)

50 thoughts on “MIT AI: OpenAI Meta-Learning and Self-Play (Ilya Sutskever)”

1. Favourite so far.

2. All the best people in AI on you course!

3. ??

4. this is gold

5. I appreciate the reminder that digital representations of ANN are really digital circuits.

6. This was a-m-a-z-i-n-g

7. Great talk!,thanks for posting

8. Pete Sampras of AI.

9. Try A* SOM

10. Найс

11. Wow really cool and summarized in a profound compact way! Thanks for talking and sharing this online.

12. your dota bot got fucked up by normal players.

13. This is great, thanks a lot.

14. This is a talk by Ilya Sutskever for course 6.S099: Artificial General Intelligence. He is the Co-Founder of OpenAI. This class is free and open to everyone. Our goal is to take an engineering approach to exploring possible paths toward building human-level intelligence for a better world.

15. This should have way more views. Grand talk.

16. Not all heroes wear capes.. Ilya is one of the most underrated thinkers in AI right now.

17. Very true and insightful..we reward ourselves, environment doesn't

18. Awesome! So many topics clearly and concisely explained.

19. off policy learning 100 likes

20. Great video!

21. joffery is that you?

22. At 44:00 he says backpropagation solves circuit search. What problem is he talking about? Anyone got references to this backpropagation and circuit search thing?

23. some good insight on DL from Ilya !

24. good talk on elaborating on some home truths.

    thanks for the vid 🙂

25. Thanx guys! Great presentation!

26. "The only real reward is existence and non-existence. Everything else is a corollary of that". Damn. That's deep.

27. Reaching to learn…?

28. 8:25 "And there is only one real true reward in life, and this is existence or non-existence, and everything else is a corollary of that." OK, that was _deep_. I would say surviving is a shared necessary condition that has many implications and that it could lead to a new era of better politics, if it got the attention it deserves. And I would not say that everything else is "a corollary", but I agree to a good extent. The video is awesome, it is just that this point may be the most important, although it is one not strongly related to machine learning.

29. i dont understand how 'the shortest solution' can even be considered. it seems nonsensical. symbols mean what we define them to mean. you could define the letter x as the shortest solution to solve a problem and the size of your program is one byte.

30. woooooooooooooooooowwwwww

31. The best talk related to AGI I have seen so far.

32. Self Play reminds of dual-simplex algorithm.

33. Nice talk, but I am a bit disappointed by the speculations on strong AI. In particular the slides at 39:30 (taken from somewhere else) are incredibly misleading. I know it is supposed to be funny but still it is a mistake to show that.

34. man this guy is a fucking genius! somehow elon is like a black hole attracting the smartest people on earth to gravitate around him

35. Usually I regret watching the Q&A part of talks, but this one was excellent.

36. very insightful and breaks it down to terms even I can grasp. Thank you for this amazing video.

37. THIS IS WHAT I ALWAYS WANTED! I never knew something like this existed and thought that people simply didn't work on it or it didn't exist but it's actually real! META LEARNING! I always thought I would have to try learning how to achieve this myself after learning all the required math, but other people have already worked on it! This is really inspiring. I really hope well be able to achieve artificial general intelligence with improvements in this field.

38. On the Q&A question on backprop and biological plausibility of DL. This talk at ICLR 2018 by Blake Richards was on this topic, and very interesting. TLDW there are viable credit assignment alternatives to backprop that are more biologically plausible https://www.youtube.com/watch?v=YUVLgccVi54

39. The talk is amazing! I just heard the zombie sounds from the humanoid figures playing soccer. Or is it just imagination?

40. Thanks.

41. We should always account for the fact that all results and "emergent behavior" (i.e. learnt, not programmed) so far are results of computation, not intelligence. In other words what we see are at best automated simulations of expected (by humans) behaviors, performed by some human-designed system. Even though results are surprising and some are truly amazing, there is nothing like consciousness, self-awaress, creativity, ability to abstract and reason, logic or ability to self-motivate, all of which are aspects of human intelligence. The field should be called Automated Learning or Advanced Problem Optimization. To use the term A.I. is really a misnomer and communicates unrealistic expectations.

42. Thank you for sharing so good resources!!!!

43. Thanks for sharing ?

44. The best ever intro to AI

45. Thank you so much for posting these videos. Really appreciate how MIT has a long tradition of sharing and disseminating knowledge.

46. I want to sit in that lecture hall

47. "Computers will have an advantage in every domain." – have to ask, I imagine you mean every well defined physical domain that can be explained by immediate sensory input, right? Almost all of what we have created in recent decades has been layer upon layer of abstraction that extends far beyond our immediate physical presence. Almost certainly that trend will continue, and humans will master the abstractions that they are forced to specify to machines.

48. What does he mean by a "small circuit"?

49. Theory:
    
    0:00 introduction & supervised learning (using neural nets/deep learning)
    
    6:45 reinforcement learning (model-free (2 types) => 1. policy gradients 2. Q-learning based)
    
    12:55 meta-learning (learning to learn)
    
    Applications:
    
    16:00 HER (hindsight experience replay) algo (learn from failures)
    
    21:40 Sim2Real using meta-learning (train a policy that can adapt to different simulation params => quickly adapts to the real world)
    
    25:30 Learning a hierarchy of actions with meta-learning
    
    28:20 Limitation of meta-learning => assumption: training distribution == test distribution
    
    29:40 self-play technique (TD-Gammon, AlphaGo Zero, Dota 2 bot)
    
    37:00 can we train AGI using the self-play?
    
    39:35 learning from human feedback/conveying goals to agents (artificial leg doing salto example)

    Questions:
    
    43:00 Does human brain use backprop?
    
    45:15 dota bot question
    
    47:22 standard deviation (maximize expected reward vs minimize std dev)
    
    48:27 cooperation as motivation for the agents?
    
    49:40 open complexity theoretic problems could help AI?
    
    51:20 the most productive research trajectories towards generative language models?
    
    53:30 do you work on evolutionary strategies (for solving RL problems) in OpenAI?
    
    54:25 could you elaborate on "right goal is a political problem"?
    
    55:42 do we need a really good model of the physical world in order to have real-world capable agents?
    
    57:18 solving the problem of self-organization?
    
    58:45 follow up: self-organization in a non-competitive environment?

    my observation:
    
    42:30 It seems to me that the most difficult problem, which we will face, will be to communicate, effectively, the "right" goals to the AI in a way so that we can somewhat predict it's future behaviour, or better said it's worst case behaviour (safety implications). After all we don't want HAL 9000 type of AI's 🙂

50. seems like Elon a bit

Comments are closed.