Train Q-learning Agent with Python – Reinforcement Learning Code Project

30 thoughts on “Train Q-learning Agent with Python – Reinforcement Learning Code Project”

1. Check out the corresponding blog and other resources for this video at:
   http://deeplizard.com/learn/video/HGeI30uATws

2. hoooray exponential decay. the champion's decay

3. what i dont get is how you can have an expected reward for anything unless you're at the second to last state before game-over. nothing elese gives a reward! so how could most of those rows for each state have anything in them at all?

4. Hey I get an error, when I update the q_table.
   "IndexError: index 4 is out of bounds for axis 0 with size 4"
   Can somebody help me?

5. NOBODY:
   LITERALLY NOBODY:
   deeplizard: Don't be shy. I wanna hear from you.

6. This tutorial is incredibly clear and is the best tutorial that I have found about RL on the internet, I have learned a lot, thanks a lot for the effort in creating this and sharing the knowledge.

7. You are the best!!! EXCELLENT Explanation, EXCELLENT video! Thank you very much, I became a fan of your work!

8. ERROR : env.reset( ) must be called before calling env.step( )

9. I am getting a decrease in the reward after running the code for 10k episodes, any ideas why ?

10. After this video I'm going to become a patreon! This was an absolutely amazing series up to this point. Can't wait to finish

11. Excellent video series!

12. great video, but you forgot a parentesis in the main formula (Update Q-table)

13. Definitely the most clear and by far the best tutorial on q learning out there. Thank you!!

14. Why are there large q-values for left and top action for starting State in the First Row of q_table, as we know it can only move to right or down?
    
    Also large value for left in 5th row for [F]HFH?
    
    By the way you are the best teacher EVER!!!! Really thankyou for these lectures.

15. Thank you for awesome explanation!!! It's the greatest tutorial I have ever seen about Q-learning

16. Thanks for this series. I had learned so much with it and i'm learning with you a lot of things.

17. Thanks for the amazing work. i have a question. for a state like "S" , which is the initial state for the agent. There will be only 2 action choices, moving right or moving down. the same also applies with states found on the edges (3 action choices) and corners (2 action choices). should the Q values of those state action pairs be zeros? just cannot locate them on the final Q-table. I know there are 23 zeros on the final Q-table. Are those for state action pairs that lead to "H" and for cases mentioned above when only 2 or 3 action choices are available?

18. So I read about a term called "return from the reward". I have two questions:

    1) Is it the same thing as the Q function?
    2) To calculate the return, We need the reward of FUTURE actions. How do we get that?

19. Dear deeplizard-team,
    first of all I want to congratulate you for having put together an amazing Reinforcement Learning playlist. Your theoretical explanations are very precise, you walk us very smoothly through the source code, and your videos are both entertaining and rich on great content! By watching your videos, you can tell that you are very passionate about AI and on sharing your knowledge with fellow tech enthusiasts. Thank you for your hard work and dedication.

    I have one question though: I am unsure where exactly we specify the positive or negative rewards in the Python code? I followed your explanation and understood that the agent is basing its decisions for each state-action-pair on the q-table-values depending on the exploration-exploitation-tradeoff. But where exactly in the source code do we actually tell the agent that by stepping onto the fields with letter H, it will receive minus 1 points and for landing on F-letter fields it is "safe"? Is this information specified in the env.step-function and thus already imported from OpenAI gym's environment? I look forward to your reply.

    Thanks!

20. Hi, why do we need to set done = False when later we get done value from env.step(action)?

21. Heyyyy, why am I getting zeroes everywhere?

22. Thank you!

23. Why am I only getting zero's, I've followed the code word for word, letter for letter, number for number. But I only get zero's

24. my reward table shows all zeros. could not replicate your result.

25. 4:13. what is doing?

26. Giving likes for any non music video very rarely, but here i gotta make exception. These videos deserve thousands, you're awesome ^^

27. Amazing explanation. I was wondering if you have the code implementation for a continous task as well.

28. why dont you publish your courses on udemy or coursera?
    i would be happy to support you there 😀
    KEEP IT UP <3

29. Beautiful video, really clear and well made!
    Only a quick question: shouldn't we consider the discount when we compute the value in "rewards_current_episode" in order to get the return? Maybe with 0.99 and only 100 steps at most one could barely notice the difference, but I think in general the formula for this implementation should be like rewards_current_episode += (DISCOUNT_RATE ** (step+1)) *reward. The +1 comes from the fact that the value corresponding to the "first" episode is actually 0 due to how range(..) works.
    Hope this is a meaningful consideration, please correct me if I'm wrong 🙂

30. love you

Comments are closed.