The Future of Deep Learning Research

29 thoughts on “The Future of Deep Learning Research”

1. What about FPGA's? Why aren't people making a big deal out of them for deep learning hardware? They'd be the perfect fit; you could just update them via software! Imagining entire batches of neural networks in single clock cycles!

2. Great talk Siraj. I always get inspired after watching your video. Keep it up!

3. When your thinking is trained to be a hammer then every new encounter resembles a nail!

   !!!

4. Why don't you put all that you said in this video into a machine code?
   Just explain it to a machine! Try OpenCYC.

5. Maybe in the short term we will use computers to augment our abilities (as we have always done with technology), but machines absolutely can and will be capable of creativity that far exceeds ours. There is no task that cannot be done better by an AI.

6. Allow me to add one more option from a hardware perspective. To build on to that requires clearing up some fundamental issues. You say everything is a function. The more accurate statement is that everything can be described as a function. The key word being description. Neural network algorithms are computational descriptions on how learning can be achieved to satisfy an input-output mapping. The option I propose is trying to understand the underlying physical (thermodynamic) process that we end up describing as learning. For eg: a refrigerator taking in electrical energy to cool things down can be described computationally using an input-output function and implemented using a transistor circuit. I can also always build a refrigerator to take in power and cool things down. Both my circuit and the refrigerator are now doing the same thing computationally but only one of them will actually cool things down. So why not attack the question of general intelligence the same way? Is it possible to build a hardware system that satisfies specific thermodynamic (energy/power/work/heat) conditions so that their dynamics can now be described as learning. For fun, let's call this system a thermodynamic computer.

7. this is way strategy i used for my first BUGS project … fun its today great idea 😀 just spawn ton of mutants until cpu die and chose bestest. theere interesting ideas emerge how to peed up evolution

8. https://youtu.be/0bhtQIhGZzM

9. Thanx . Thats all the inspiration i needed…… And as for backprop ya just like u i too started to think maybe we r overusing and more and more models are arching towards it. O.o and i m just learning now.

10. Hi there from Germany! I really like your videos about deep learning and neural nets but I have to make a little critique:
    Please don't explain terminology by using the exact same word! Like when you said 'We call it back propagation, right, beause we are back propagating an error […]'
    This may seem obvious to a native english speaking person but won't help at all if you don't already know these terms…

11. chief keef. citgo

12. Autoencoder's "REAL" output is not its input, it should be its middle layer's representation. Though we train the AI system to try to generate the output as same as the input, but that is what the AI "want", not what we want. It's just like we raise a cow, and command it to bear a baby, but the baby cow is not our final target, our final target(Human's target) is the mother-process byproduct: milk.
    Is my understanding right?

13. Babies and children die without supervision. They need their parents to tell them how something works before they can actually learn doing said task. Even as children they often need you to look at the pictures they drew and „rate“ them. They need feedback all the time so they can adjust. Same thing with talking. They say something, you correct them, they say it again and so on. So in my opinion, they are not able of doing unsupervised learning without seeing how others do it or getting feedback.

14. You're the best, keep it up!

15. isnt the low energy of learning for biological brain also because of its limitation on how fast it could learn?. machine learning does need more power with the benefit of minimizing times needs for learning. for example machine learning could differentiate 100 breeds of dogs well for lets say 1 night when human need more long extensive learning for it. and of course like what siraj said, the improvement in hardware still have so much room for innovation, like alternative energy, new more efficient architecture etc.

16. 1,000,000 * 1,000,000 = 1,000,000,000,000

17. The name of video is a bit misleading. I realized after i was 20 mins thru meaning i lost tat time 🙁

18. No words for ur video.Excellent

19. Within back propagation, is there a way to prune the hidden nodes? If a node isn't actually relevant, can it be eliminated automatically? I'm thinking of an analogy to the brain where layers are sparse, and a computer equivalent would be more efficient if it needed less memory/processing for unnecessary intermediate nodes.

    I wish I had more time in my day. I'm one of those people that started in the 70's with AI, but my career took me in a different direction.

    Love your videos. Keep it up.

20. You make go crazy again….emotionally motivating. Good for my neural network.

21. But unsupervised and supervised learning are sometimes tight together. For example prediction of future is classification on unlabeled data (you just want to predict next state of system using unlabeled sequence of states in time). And I think, thats what the brain does.

22. But why partial derivative wrt weights can i understand in simple logic.. please

23. it's was mathematically proven that kohonen and kmeans are the same……..mmmmmmm, ai reinvented the wheel!
    and it's the case with lots of ai tools
    the problem is, if you say ' i used ai and kohonen' you are supposed to be a god, if you say ' i used a kmeans' you are just told you are an old fashion moron
    times are tough, huh?

24. He didn't invent backpropagation – see Schmidhuber

25. very good show!

26. Thank you 🙂

27. Artificial Life is the closest field studying the topic that you concluded as the most promising direction. It's an exciting field! I would love to see more interactions between AI and AL. Siraj, this could possibly be an interesting topic for a video?

28. Poor chroma key config

29. it is simple how babies learns :
    1- there is the supervision of instincts
    2- using a feedback loop

Comments are closed.