Activation Functions in a Neural Network explained

29 thoughts on “Activation Functions in a Neural Network explained”

1. *Note, starting at 1:09, the denominator of the sigmoid function should be e^x+1 rather than e^(x+1).*

   Machine Learning / Deep Learning Tutorials for Programmers playlist: https://www.youtube.com/playlist?list=PLZbbT5o_s2xq7LwI2y8_QtvuXZedL6tQU

   Keras Machine Learning / Deep Learning Tutorial playlist: https://www.youtube.com/playlist?list=PLZbbT5o_s2xrwRnXk_yCPtnqqo4_u2YGL

2. Which programming language are you using?

3. Thanks a lot mam! This helped a lot

4. In relu, More positive it is, more activated the node is. For, 1 it'll be activated, for 3 it'll be activated also..! Than what's the difference between different positive values? Node is gonna activated anyway

5. Amazing tutorials!

6. Thank you so much

7. 1:11 where does the six come from? is it a trainable value like a bias, or is it the number of nodes in the active layer, or is it always six? In other words, why is the sigmoid's input limited to six?

8. first and foremost,
   thank you for amazing explain,

9. Hi, I would like to know which of the independent variables is much significant or has the highest impact on the dependent variable. My model is a 9-5-1 MLP which I have extracted its weights and biases. However, my concern now is how to use those weights to determine the most relevant input to the least relevant so I can rank. Thank you.

10. You're my favorite Machine Learning teacher, please keep making such videos.

11. You never explain WHY to use an activation function. You just showed us HOW it works.

12. great video Tnx!

13. Nice Explanation. What is softmax and why it is widely used in Deep learning instead of general activation functions?

14. You saved my tommorow's ML exam.

15. I've just started ML and thanks to you. Cuz of you I'm able to digest Coursera lectures practically. And finally you've earned a subscriber ?

16. Simple, but it helped me out a lot!

17. Thank you so much 🙂 Keep making such amazing videos please.

18. Does Deep learning with Keras need no skills?
    One of my friend said that it's irrelevant to code with Keras :'-(

19. Hi, I found this from somewhere "Because of the horizontal line in ReLu for when x<0, the gradient can go towards 0 . For activations in that region of ReLu, gradient will be 0 causing the weights to not get adjusted during descent. Hence, those neurons die and not respond making part of the NN passive"

    I don't understand why a gradient of 0, (gradient in this context seems to refer to the gradient of the relu function rather than the gradient of the loss function) will not allow for weights to get adjusted. Please help!

20. thank you very much

21. This is totally wrong way to explain why we need activation function. We hide behind the biological explanation of neurons to explain why activation function is needed.

22. The answer lies in linearity and non linearity.

23. Thank you for these videos! Its so easy to follow your explanations.

24. Activation function happens on the second half of a node. So it might be miss leading to show all the weight pointing to the next hidden layer.

25. without a doubt you are one of the best instructor I have ever seen. I absolutely love your channel. well done! and thank you so much for your great help. You saved my life

26. Perfect Explanation .

27. Extremely! Wonderful Explanation <3 it

28. Love your voice

29. can i access this jupyter notebook?

Comments are closed.