Lex Fridman
Vladimir Vapnik is the co-inventor of support vector machines, support vector clustering, VC theory, and many foundational ideas in statistical learning. He was born in the Soviet Union, worked at the Institute of Control Sciences in Moscow, then in the US, worked at AT&T, NEC Labs, Facebook AI Research, and now is a professor at Columbia University. His work has been cited over 170,000 times. He has some very interesting ideas about artificial intelligence and the nature of learning, especially on the limits of our current approaches and the open problems in the field.
This conversation is part of the Artificial Intelligence podcast and the MIT course 6.S099: Artificial General Intelligence. The conversation and lectures are free and open to everyone. Our goal is to take an engineering approach to exploring possible paths toward building human-level intelligence for a better world. Audio podcast version is available on https://lexfridman.com/ai/
INFO:
Podcast website: https://lexfridman.com/ai
Course website: https://agi.mit.edu
YouTube Playlist: http://bit.ly/2EcbaKf
CONNECT:
– Subscribe to this YouTube channel
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Twitter: https://twitter.com/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– Slack: https://deep-mit-slack.herokuapp.com
Source
0:00 Introduction by Prof. Lex
1:04 Fundamental nature of reality : Does god play dice ? (Refers Albert Einstein)
1:54 Philosophy of science : Instrumentalism and Realism
4:08 The unreasonable effectiveness of mathematics [1][2]
6:08 Math and simple underlying principles of reality
7:26 Human intuition and ingenuity
8:56 Role of imagination (Refers Einstein's special relativity)
10:00 Do we/ will have tools to describe the process of learning mathematically ? (Refers Hook's Microscope) [3][4][5]
12:16 From a Mathematical point of view : What is a great Teacher ?
13:48 Mechanism in Learning and Essence of Duck (Bumper sticker material. Quack Quack !!)
16:58 How far are we from integrating the predicates ? (Refer the duck content to understand this question)
18:17 Admissible Set of Functions and Predicates (Talks about VC Theory [6])
23:01 What do you think about deep learning ? (Mentions Churchill's book "The Second World War" [7], Shallow Learning [8])
27:57 Alpha Go and Effectiveness of Neural Networks [9]
30:46 Human Intelligence and Alan Turing
33:34 Big-O Complexity and Worst Case Analysis
38:49 Opinion of how AI is considered as coding to imitate a human being
39:44 Learning and intelligence
42:09 Interesting problems on Statistical Learning (Mentions Digit Recognition problem and importance of intelligence)
48:48 Poetry, Philosophy and Mathematics
50:40 Happiest Moment as a Researcher
References :
[1] Wigner, Eugene P. "The unreasonable effectiveness of mathematics in the natural sciences." In Mathematics and Science, pp. 291-306. 1990.
[2] http://www.hep.upenn.edu/~johnda/Papers/wignerUnreasonableEffectiveness.pdf
[3] https://youtu.be/2gtrkxtsQ2k
[4] https://books.google.com/books?hl=en&lr=&id=ISP_gRwuz94C&oi=fnd&pg=PR1&dq=Micrographia+hook&ots=LF1VWdxjQg&sig=Qca7QzxkynZXc4AGy0YldNdQP_k
[5] Hook, Robert. "Micrographia: Or Some Physiological Descriptions of Minute Bodies Made by Magnifying Glasses with Observation and Inquiries Thereupon." Royal Society: London, UK 1665.
[6] https://www.cs.cmu.edu/~bapoczos/Classes/ML10715_2015Fall/slides/VCdimension.pdf
[7] https://www.goodreads.com/book/show/25587.The_Second_World_War
[8] https://files.meetup.com/18405165/DLmeetup.pdf
[9] https://www.imdb.com/title/tt6700846/
Thank you for uploading such a beautiful interview! I enjoyed this video so much!
With each interview, I'm getting more interested in the subject. Thank you for the great content!
really interesting conversation, thank you!
His comment about music is similar to the ideas in GEB!
I haven't watched the entire video as of now, but on the same subject can you get Chomsky and Norvig.
It would be great if we had similar lecture also in Russian
Ground Truths guide us all <3
I am not sure if we can derive theory of inteligence purely from math. In physics the problems are easier, because we can create meaningful equations, which can guide us. The examples could be Max plank quantization of energy or Albert Einstein retativity theory or Dirac's anti particles or currently string theory.
On the other hand in biology, chemistry, … there is less insight from equations. For example effects of protein folding are very difficult to deduce from equations and we have to use computation instead. The same could be with intelligence that it has mathematical description, but is very messy and does not adhere to our sense of mathematical beaty. This could of course change as we find more connections and built consistant theory, so initially messy ideas become more and more intuitive and beautiful, but the core does not change.
Using beauty and elegance of math as heuristic is a little bit dangerous. For example geocentric theory at the time had nicer description than heliocentric theory. The reason was that we had to made more correction term to heliocentric theory to match the precision of geocentric theory. It was, because they didn't use elipse to describe motion, but instead compositions of circular motions were used. Only after emprical findings of Kepler we switched to elipses.
Another more anecdotal example would be the dynamo theory of WALTER M. ELSÄSSER describing why plantes have magnetic fields. He told his theory to Albert Einstein, but “he didn’t
much believe it. He simply could not believe that something so beautiful could have such a complicated explanation" in words of Einsten assistan (Einstein prefered not to tell his opinion). The theory was correct, Einstein's intuition was wrong. (Source: top of 3rd page of pdf -> http://www.geosociety.org/documents/gsa/memorials/v24/Elsasser-WM.pdf)
Also currently string theory is getting some backlash, because of lack of results despite decade long effort. This theory has some promising connections and seems to be a perfect fit for missing element in our understanding of physics, but there are also some ugly parts, like need for more dimensions or too many possible universes.
So we have to be carefull to not be too much focused on mathematical beauty, nature can just be messy or we might not have a mathematical tools to appreciate it's beauty.
I strongly disagree with Vapnik on his opinion about intuition. He seems dogmatic in his dismissal of the idea, however, through history we have seen a number of human phenotypes that produce significant intellectual achievement. One such phenotype that appears to be convergent in many individuals who have made tremendous achievements and cracked open entire academic disciplines (e.g. Einstein) is that of the visionary. Someone who is able to intimately understand a problem so that they may sufficiently abstract it to allow for giant leaps of progress by using intuition or visualization rather than iterative logical steps. I feel like Vapnik may be more of the literal, autistic type of individual who is very good at specializing and using brute force logic to iterate from axioms to a model within his discipline.
I would not be too quick to discount the role of intuition particularly in the more demanding, technical fields such as pure mathematics and theoretical physics as opposed to machine learning and statistics.
Hey Lex, Thanks for making this content free and accessible online! Very generous and much appreciated.
haha I liked his response to the AlphaGo question!
On the other hand, I think it's missleading. Just like in maths, a problem's difficulty should be gauged by how hard it seems before solving it, not how hard it is in hinsight.
I have to express my gratitude for uploading stuff like this, Thanks so much Lex and thanks to Dr. Vapnik for taking the time to express some of the insights he has gained throughout his life
Beauty and poetry! Again, thanks Lex!
very interesting person
Wow what an interesting conversation, thank you so much Lex for the video, really appreciate it and looking forward to more of such videos, cheers
?
This was incredible.
Just another day I was thinking about "how come ideas are generated in different parts of the world within a definite time period simultaneously?". Glad to hear that a prominent mathematician thinks the same way (31:34).
It's Platonic and poetic. And I have heard many mathematicians say this sort of thing. Ramanujan is also a great example that makes this theory interesting.
I can't remember the time that I've really enjoyed a great conversation like this one.These are good questions by Lex . And I am so excited and thrilled by the intelligence of Vladimir Vapnik.
So in a way, the problem of intelligence or at least the basis regarding the concept of a good teacher hinges on metaphorical truth and linguistic precision.