Open AI GPT-2 Beginners Tutorial – The Engineering of Conscious Experience

Garrett Pete

Fine tuning tutorial: https://www.youtube.com/watch?v=4iK-IuvatxI
Github: https://github.com/openai/gpt-2
Make sure to install python3, pip, and git using either apt-get or brew install.

Hope you learn some useful new stuff!

<iframe></p> <p><a href="https://www.youtube.com/watch?v=h_PIeZ9eHuo">Source</a></p> <div class="be1e40beae42d993bafb8643f4ddde8b" data-index="3" style="float: none; margin:10px 0 10px 0; text-align:center;"> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-9244112244416304" data-ad-slot="4549240677"></ins> <script> (adsbygoogle = window.adsbygoogle || []).push({}); </script> </div> <div style="font-size: 0px; height: 0px; line-height: 0px; margin: 0; padding: 0; clear: both;"></div> </div> </article> <div class="clearfix"></div> <ul class="default-theme-post-navigation"> <li class="theme-nav-previous"><a href="https://theengineeringofconsciousexperience.com/simulation-9-klee-irwin-grand-unification-theory-consciousness-cosmos/" rel="prev"><span class="meta-nav">←</span> Simulation #9 Klee Irwin – Grand Unification Theory, Consciousness, Cosmos</a></li> <li class="theme-nav-next"><a href="https://theengineeringofconsciousexperience.com/wie-is-daniel-dennett-college-door-filosoof-marc-slors/" rel="next">Wie is Daniel Dennett? College door filosoof Marc Slors <span class="meta-nav">→</span></a></li> </ul> <div class="clearfix"></div> <h3 class='comment-reply-title'>Similar Posts</h3> <div class="mb-related-posts mb-simple-featured-posts mb-simple-featured-posts-wrap row"> <article class="mb-featured-article col-md-4 px-lg-3 post"> <a class="post-thumbnail" href="https://theengineeringofconsciousexperience.com/inspecting-neural-networks-with-cca-a-gentle-intro-explainable-ai-for-deep-learning/" aria-hidden="true" tabindex="-1"> <img width="501" height="282" src="https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/06/1623359877_maxresdefault.jpg" class="attachment-magazinebook-featured-image-medium size-magazinebook-featured-image-medium wp-post-image" alt="" decoding="async" loading="lazy" srcset="https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/06/1623359877_maxresdefault.jpg 1280w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/06/1623359877_maxresdefault-300x169.jpg 300w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/06/1623359877_maxresdefault-1024x576.jpg 1024w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/06/1623359877_maxresdefault-768x432.jpg 768w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/06/1623359877_maxresdefault-520x293.jpg 520w" sizes="(max-width: 501px) 100vw, 501px" /> </a> <span class="cat-links"><a href="https://theengineeringofconsciousexperience.com/category/gpt-3/" rel="category tag">GPT 3</a></span> <header class="entry-header"> <h3 class="entry-title"><a href="https://theengineeringofconsciousexperience.com/inspecting-neural-networks-with-cca-a-gentle-intro-explainable-ai-for-deep-learning/" rel="bookmark">Inspecting Neural Networks with CCA – A Gentle Intro (Explainable AI for Deep Learning)</a></h3> <div class="entry-meta"> <span class="posted-on"><i class="far fa-calendar-alt"></i><a href="https://theengineeringofconsciousexperience.com/inspecting-neural-networks-with-cca-a-gentle-intro-explainable-ai-for-deep-learning/" rel="bookmark"><time class="entry-date published updated" datetime="2021-06-08T23:45:05-07:00">June 8, 2021</time></a></span><span class="byline"><i class="far fa-user-circle"></i><span class="author vcard"><a class="url fn n" href="https://theengineeringofconsciousexperience.com/author/a77c326ce1bca029955b922e9ba4fb94/">Jay Alammar</a></span></span> </div> </header> </article> <article class="mb-featured-article col-md-4 px-lg-3 post"> <a class="post-thumbnail" href="https://theengineeringofconsciousexperience.com/mahesh-dalle-mamtas-interlude-official-full-music-video/" aria-hidden="true" tabindex="-1"> <img width="480" height="300" src="https://theengineeringofconsciousexperience.com/wp-content/uploads/2023/11/1699050990_hqdefault-480x300.jpg" class="attachment-magazinebook-featured-image-medium size-magazinebook-featured-image-medium wp-post-image" alt="" decoding="async" loading="lazy" /> </a> <span class="cat-links"><a href="https://theengineeringofconsciousexperience.com/category/gpt-3/" rel="category tag">GPT 3</a></span> <header class="entry-header"> <h3 class="entry-title"><a href="https://theengineeringofconsciousexperience.com/mahesh-dalle-mamtas-interlude-official-full-music-video/" rel="bookmark">Mahesh dalle | Mamta’s Interlude | official full Music Video</a></h3> <div class="entry-meta"> <span class="posted-on"><i class="far fa-calendar-alt"></i><a href="https://theengineeringofconsciousexperience.com/mahesh-dalle-mamtas-interlude-official-full-music-video/" rel="bookmark"><time class="entry-date published updated" datetime="2023-06-15T10:52:21-07:00">June 15, 2023</time></a></span><span class="byline"><i class="far fa-user-circle"></i><span class="author vcard"><a class="url fn n" href="https://theengineeringofconsciousexperience.com/author/51140aad1d103396a80f8a5e7d760c27/">『$UNNŸ』</a></span></span> </div> </header> </article> <article class="mb-featured-article col-md-4 px-lg-3 post"> <a class="post-thumbnail" href="https://theengineeringofconsciousexperience.com/shot-in-the-dark-by-ac-dc-drum-cover/" aria-hidden="true" tabindex="-1"> <img width="400" height="300" src="https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/10/1602813275_hqdefault.jpg" class="attachment-magazinebook-featured-image-medium size-magazinebook-featured-image-medium wp-post-image" alt="" decoding="async" loading="lazy" srcset="https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/10/1602813275_hqdefault.jpg 480w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/10/1602813275_hqdefault-300x225.jpg 300w" sizes="(max-width: 400px) 100vw, 400px" /> </a> <span class="cat-links"><a href="https://theengineeringofconsciousexperience.com/category/gpt-3/" rel="category tag">GPT 3</a></span> <header class="entry-header"> <h3 class="entry-title"><a href="https://theengineeringofconsciousexperience.com/shot-in-the-dark-by-ac-dc-drum-cover/" rel="bookmark">Shot In The Dark by AC/DC (Drum Cover)</a></h3> <div class="entry-meta"> <span class="posted-on"><i class="far fa-calendar-alt"></i><a href="https://theengineeringofconsciousexperience.com/shot-in-the-dark-by-ac-dc-drum-cover/" rel="bookmark"><time class="entry-date published updated" datetime="2020-10-08T10:20:30-07:00">October 8, 2020</time></a></span><span class="byline"><i class="far fa-user-circle"></i><span class="author vcard"><a class="url fn n" href="https://theengineeringofconsciousexperience.com/author/019b271d2e1e5c5353fec9ff97ff387f/">Adam Lippy</a></span></span> </div> </header> </article> </div> <div id="comments" class="comments-area"> <h5 class="comments-title"> 26 thoughts on “<span>Open AI GPT-2 Beginners Tutorial</span>” </h5> <ol class="comment-list"> <li id="comment-179806" class="comment even thread-even depth-1"> <article id="div-comment-179806" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCzAZD4IvRMqH5Ov5owoOarQ" class="url" rel="ugc external nofollow">cole slay</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179806"><time datetime="2019-10-04T00:02:14-07:00">October 4, 2019 at 12:02 am</time></a> </div> </footer> <div class="comment-content"> <p>What specs are your laptop? I read your article and want to know how much of my resources I should allocate to this. Should I run it through a virtual machine or my home environment? Thanks</p> </div> </article> </li> <li id="comment-179805" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-179805" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC32r8clw_qMtDM5zvkGoGtQ" class="url" rel="ugc external nofollow">Chris M</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179805"><time datetime="2019-11-15T08:02:30-07:00">November 15, 2019 at 8:02 am</time></a> </div> </footer> <div class="comment-content"> <p>Hey, thanks for your video. Please could I ask for some advice. Im not a programmer but am really interested in ML. I've met all requirements but cant seem to reconfigure the generate_unconditional_<a href="http://samples.py/">samples.py</a> to run the big model. Im not sure what Im doing wrong, I changed the default model to 1558M in the definitions and the path but I dont know the correct syntax</p> </div> </article> </li> <li id="comment-179804" class="comment even thread-even depth-1"> <article id="div-comment-179804" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCrAbGFfXyuFNIPZVbNJOE6g" class="url" rel="ugc external nofollow">Aleksandar Vujanov</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179804"><time datetime="2019-11-22T11:56:15-07:00">November 22, 2019 at 11:56 am</time></a> </div> </footer> <div class="comment-content"> <p>I would be interested if you could leave your email, or any kind of contact, so I can get in touch with you.<br />I'd like to hire you to do it for me, and tweak it a bit, for a purpose I have in my mind.</p> <p>Thanks in advance</p> </div> </article> </li> <li id="comment-179803" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-179803" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCfqMKVwH4PvgXrWJN7s3csA" class="url" rel="ugc external nofollow">Johnny9</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179803"><time datetime="2019-11-25T10:16:18-07:00">November 25, 2019 at 10:16 am</time></a> </div> </footer> <div class="comment-content"> <p>does it just write random stories or can you get it to rewrite passages?</p> </div> </article> </li> <li id="comment-179802" class="comment even thread-even depth-1"> <article id="div-comment-179802" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCk9PFz8uLJAfnxOzRnQc9nw" class="url" rel="ugc external nofollow">Adam O'Dwyer</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179802"><time datetime="2020-01-16T13:12:39-07:00">January 16, 2020 at 1:12 pm</time></a> </div> </footer> <div class="comment-content"> <p>Coincidentally I am at that same point with the same numpy issue once I pasted "python3 src/interactive_conditional_<a href="http://samples.py/">samples.py</a> –top_k 40" <br />I did notice that when you posted this vid, you showed 3 python models. Now there are 4 models. The last one is a 1.558m file. Might that be the issue?</p> </div> </article> </li> <li id="comment-179801" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-179801" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCCgm5XQACKdj_XKxNULZNcA" class="url" rel="ugc external nofollow">Levi Heizer</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179801"><time datetime="2020-03-04T12:29:06-07:00">March 4, 2020 at 12:29 pm</time></a> </div> </footer> <div class="comment-content"> <p>I have a question on this. If you just run the model like the way you do in the video does it develop in any way? Is it training input to input at all or just starting over each time?</p> </div> </article> </li> <li id="comment-179800" class="comment even thread-even depth-1"> <article id="div-comment-179800" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCFFHXq7KMFzxPi1Ez3dld8g" class="url" rel="ugc external nofollow">Catffee</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179800"><time datetime="2020-03-12T23:17:08-07:00">March 12, 2020 at 11:17 pm</time></a> </div> </footer> <div class="comment-content"> <p>Help<br /> <br />if i throw<br /> <br />pip install tqdm == 4.31.1<br /> <br />Result: Requirement already satisfied: tqdm == 4.31.1 in c: programdata miniconda3 lib site-packages (4.31.1)<br /> <br />but if I throw<br /> <br />python3 download_<a href="http://model.py/">model.py</a> 124M<br /> <br />result: from tqdm import tqdm<br /> <br />ModuleNotFoundError: No module named 'tqdm'</p> </div> </article> </li> <li id="comment-179799" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-179799" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCj_VHg5_1KabXYXJ0cKTM6Q" class="url" rel="ugc external nofollow">Dultimate</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179799"><time datetime="2020-03-29T22:00:20-07:00">March 29, 2020 at 10:00 pm</time></a> </div> </footer> <div class="comment-content"> <p>Hi, I jokingly decided to use GTP-2 and followed the instructions on this tutorial but I need help, on the last step where I put the sudo command for interactive conditional samples it keeps ending with an AttributeError: "module 'TensorFlow' has no attribute 'sort' ", I've installed the non GPU tensor flow through the command and don't know what I need to do to fix this, I'm using a old weak laptop with a 4th gen i3 if that's any help</p> </div> </article> </li> <li id="comment-179798" class="comment even thread-even depth-1"> <article id="div-comment-179798" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCXDurZOPNpD0eD2noYkhTOQ" class="url" rel="ugc external nofollow">Elie Skg</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179798"><time datetime="2020-04-19T16:26:13-07:00">April 19, 2020 at 4:26 pm</time></a> </div> </footer> <div class="comment-content"> <p>Hi, I basically just followed your video and don't know anything about programing but after installing the sudo python3 download_<a href="http://model.py/">model.py</a> 124M command, I got stuck trying to run this command:</p> <p> sudo python3 src/interactive_conditional_<a href="http://samples.py/">samples.py</a> –top_k 40</p> <p>The following appeared:<br />Traceback (most recent call last):</p> <p> File "src/interactive_conditional_<a href="http://samples.py/">samples.py</a>", line 6, in <module></p> <p> import numpy as np</p> <p>ModuleNotFoundError: No module named 'numpy'</p> <p>Would you know what I am supposed to do ?</p> <p>UPDATE:</p> <p>I have updated the TensorFlow and now this appears:</p> <p>MacBook-Pro-de-Elie:gpt-2 elie$ sudo python3 src/interactive_conditional_<a href="http://samples.py/">samples.py</a> –top_k 40</p> <p>Password:</p> <p>Traceback (most recent call last):</p> <p> File "src/interactive_conditional_<a href="http://samples.py/">samples.py</a>", line 7, in <module></p> <p> import tensorflow as tf</p> <p> File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/tensorflow/__init__.py", line 41, in <module></p> <p> from <a href="http://tensorflow.python.tools/">tensorflow.python.tools</a> import module_util as _module_util</p> <p> File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/tensorflow/python/__init__.py", line 50, in <module></p> <p> from tensorflow.python import pywrap_tensorflow</p> <p>ImportError: cannot import name 'pywrap_tensorflow' from partially initialized module 'tensorflow.python' (most likely due to a circular import) (/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/tensorflow/python/__init__.py)</p> <p>I am Still stuck!!</p> </div> </article> </li> <li id="comment-179797" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-179797" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCMzp2A6bV9zh7WdTh04IKsg" class="url" rel="ugc external nofollow">Ethan Gibbs</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179797"><time datetime="2020-05-03T06:30:42-07:00">May 3, 2020 at 6:30 am</time></a> </div> </footer> <div class="comment-content"> <p>soodoo</p> </div> </article> </li> <li id="comment-179796" class="comment even thread-even depth-1"> <article id="div-comment-179796" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCUmCGHdJqsxYl4ehVMy6TOw" class="url" rel="ugc external nofollow">Muhammad Salimov</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179796"><time datetime="2020-05-15T00:30:58-07:00">May 15, 2020 at 12:30 am</time></a> </div> </footer> <div class="comment-content"> <p>after running this command 'sudo python3 src/interactive_conditional_<a href="http://samples.py/">samples.py</a> –top_k 40' I've got the following issue: Traceback (most recent call last):</p> <p> File "src/interactive_conditional_<a href="http://samples.py/">samples.py</a>", line 6, in <module></p> <p> import numpy as np</p> <p>ModuleNotFoundError: No module named 'numpy'</p> <p>muhammadsalimov@MUHAMMADs-MacBook-Pro gpt-2 %</p> </div> </article> </li> <li id="comment-179795" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-179795" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCUmCGHdJqsxYl4ehVMy6TOw" class="url" rel="ugc external nofollow">Muhammad Salimov</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179795"><time datetime="2020-05-15T07:59:16-07:00">May 15, 2020 at 7:59 am</time></a> </div> </footer> <div class="comment-content"> <p>What version of python do you have?</p> </div> </article> </li> <li id="comment-179794" class="comment even thread-even depth-1"> <article id="div-comment-179794" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCwv9kvr1kIM-BAWxoVMS3GQ" class="url" rel="ugc external nofollow">Mar2ck</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179794"><time datetime="2020-06-02T14:07:13-07:00">June 2, 2020 at 2:07 pm</time></a> </div> </footer> <div class="comment-content"> <p>Why would you run git clone as sudo???</p> </div> </article> </li> <li id="comment-179793" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-179793" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCwv9kvr1kIM-BAWxoVMS3GQ" class="url" rel="ugc external nofollow">Mar2ck</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179793"><time datetime="2020-06-02T16:41:59-07:00">June 2, 2020 at 4:41 pm</time></a> </div> </footer> <div class="comment-content"> <p>I got it to work with python 3.7 + tensorflow 1.15.2</p> </div> </article> </li> <li id="comment-179792" class="comment even thread-even depth-1"> <article id="div-comment-179792" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCxGG4eyukXe5h8JUgdzFl3w" class="url" rel="ugc external nofollow">Carlos Cloud & Web Developer</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179792"><time datetime="2020-06-11T07:22:23-07:00">June 11, 2020 at 7:22 am</time></a> </div> </footer> <div class="comment-content"> <p>Any ideas as to how to bring that generated text to a react front end?</p> </div> </article> </li> <li id="comment-179791" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-179791" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCYLtxbPZkYG2m0CYZkxIkDQ" class="url" rel="ugc external nofollow">OneKe</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179791"><time datetime="2020-06-16T09:15:29-07:00">June 16, 2020 at 9:15 am</time></a> </div> </footer> <div class="comment-content"> <p>Traceback (most recent call last):<br /> File "src/interactive_conditional_<a href="http://samples.py/">samples.py</a>", line 3, in <module><br /> import fire<br />ModuleNotFoundError: No module named 'fire'<br />What the heck</p> </div> </article> </li> <li id="comment-179790" class="comment even thread-even depth-1"> <article id="div-comment-179790" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCVgpaddSVzw4wGX00hf6l6g" class="url" rel="ugc external nofollow">pierrick sylvestre</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179790"><time datetime="2020-06-21T13:26:12-07:00">June 21, 2020 at 1:26 pm</time></a> </div> </footer> <div class="comment-content"> <p>Thanks for this tutorial, Garrett. But like many others here, I am stuck with the "numpy" issue. I believe your tutorial is the most recent one on YT so I was hoping you could look into it. I wanna run GPT-2 so badly…..!!!</p> </div> </article> </li> <li id="comment-179789" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-179789" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCLS_xzCIHVXSO2OlN8CAE5A" class="url" rel="ugc external nofollow">MirabelleJones</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179789"><time datetime="2020-07-12T12:23:55-07:00">July 12, 2020 at 12:23 pm</time></a> </div> </footer> <div class="comment-content"> <p>For anyone who is having pipeline issues, make sure you have Python 3.7 – 64 bit (not 32) installed and are using tensorflow 1.15. Tensorflow 2.x (anything after 2.0) does not include the same modules. Similarly, this will not work with the newest version of the 64 bit architecture – Python (3.9).</p> </div> </article> </li> <li id="comment-179788" class="comment even thread-even depth-1"> <article id="div-comment-179788" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCMcJOlBThGEWWqb3vbJlbTA" class="url" rel="ugc external nofollow">Raphael</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179788"><time datetime="2020-07-18T18:09:34-07:00">July 18, 2020 at 6:09 pm</time></a> </div> </footer> <div class="comment-content"> <p>thanks</p> </div> </article> </li> <li id="comment-179787" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-179787" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCViBfWIXwTq7CRKmHImR3NA" class="url" rel="ugc external nofollow">INVALID USER</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179787"><time datetime="2020-07-20T08:27:14-07:00">July 20, 2020 at 8:27 am</time></a> </div> </footer> <div class="comment-content"> <p>Does this work on Linux?</p> </div> </article> </li> <li id="comment-179786" class="comment even thread-even depth-1"> <article id="div-comment-179786" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCSagXAUALBJ0NQK2JIGdpMQ" class="url" rel="ugc external nofollow">Komlan Cohovi</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179786"><time datetime="2020-07-24T02:58:49-07:00">July 24, 2020 at 2:58 am</time></a> </div> </footer> <div class="comment-content"> <p>I got permission error is that because of my anaconda?</p> <p>Successfully installed fire-0.3.1 regex-2017.4.5 requests-2.21.0 termcolor-1.1.0 tqdm-4.31.1 urllib3-1.24.3<br />(base) Dodjis-MacBook-Pro:gpt-2 dodjicohovi$ sudo python3 download_<a href="http://model.py/">model.py</a> 124M<br />Fetching checkpoint: 1.00kit [<a href="https://www.youtube.com/watch?v=h_PIeZ9eHuo&t=00m00s">00:00</a>, 479kit/s] <br />Fetching encoder.json: 1.04Mit [<a href="https://www.youtube.com/watch?v=h_PIeZ9eHuo&t=00m00s">00:00</a>, 6.96Mit/s] <br />Fetching hparams.json: 1.00kit [<a href="https://www.youtube.com/watch?v=h_PIeZ9eHuo&t=00m00s">00:00</a>, 501kit/s] <br />Fetching model.ckpt.data-00000-of-00001: 498Mit [<a href="https://www.youtube.com/watch?v=h_PIeZ9eHuo&t=00m21s">00:21</a>, 22.9Mit/s] <br />Fetching model.ckpt.index: 6.00kit [<a href="https://www.youtube.com/watch?v=h_PIeZ9eHuo&t=00m00s">00:00</a>, 2.79Mit/s] <br />Fetching model.ckpt.meta: 472kit [<a href="https://www.youtube.com/watch?v=h_PIeZ9eHuo&t=00m00s">00:00</a>, 5.09Mit/s] <br />Fetching vocab.bpe: 457kit [<a href="https://www.youtube.com/watch?v=h_PIeZ9eHuo&t=00m00s">00:00</a>, 3.69Mit/s] <br />(base) Dodjis-MacBook-Pro:gpt-2 dodjicohovi$ python3 download_<a href="http://model.py/">model.py</a> 355M<br />Traceback (most recent call last):<br /> File "download_<a href="http://model.py/">model.py</a>", line 14, in <module><br /> os.makedirs(subdir)<br /> File "/Users/dodjicohovi/anaconda3/lib/python3.7/<a href="http://os.py/">os.py</a>", line 221, in makedirs<br /> mkdir(name, mode)<br />PermissionError: [Errno 13] Permission denied: 'models/355M'</p> <p>I believe the version of the tf is not compatible. I don't know what I am doing </p> <p>Traceback (most recent call last):</p> <p> File "src/interactive_conditional_<a href="http://samples.py/">samples.py</a>", line 7, in <module></p> <p> import tensorflow as tf</p> <p>ModuleNotFoundError: No module named 'tensorflow'</p> <p>(base) Dodjis-MacBook-Pro:gpt-2 dodjicohovi$ pip3 install tensorflow==1.12.0</p> <p>ERROR: Could not find a version that satisfies the requirement tensorflow==1.12.0 (from versions: 1.13.0rc1, 1.13.0rc2, 1.13.1, 1.13.2, 1.14.0rc0, 1.14.0rc1, 1.14.0, 1.15.0rc0, 1.15.0rc1, 1.15.0rc2, 1.15.0rc3, 1.15.0, 1.15.2, 1.15.3, 2.0.0a0, 2.0.0b0, 2.0.0b1, 2.0.0rc0, 2.0.0rc1, 2.0.0rc2, 2.0.0, 2.0.1, 2.0.2, 2.1.0rc0, 2.1.0rc1, 2.1.0rc2, 2.1.0, 2.1.1, 2.2.0rc0, 2.2.0rc1, 2.2.0rc2, 2.2.0rc3, 2.2.0rc4, 2.2.0, 2.3.0rc0, 2.3.0rc1, 2.3.0rc2)</p> <p>ERROR: No matching distribution found for tensorflow==1.12.0</p> <p>(base) Dodjis-MacBook-Pro:gpt-2 dodjicohovi$ pip3 install tensorflow-gpu==1.12.0</p> <p>ERROR: Could not find a version that satisfies the requirement tensorflow-gpu==1.12.0 (from versions: none)</p> <p>ERROR: No matching distribution found for tensorflow-gpu==1.12.0</p> <p>(base) Dodjis-MacBook-Pro:gpt-2 dodjicohovi$ sudo python3 src/interactive_conditional_<a href="http://samples.py/">samples.py</a> –top_k 40</p> <p>Traceback (most recent call last):</p> <p> File "src/interactive_conditional_<a href="http://samples.py/">samples.py</a>", line 9, in <module></p> <p> import model, sample, encoder</p> <p> File "/Users/dodjicohovi/Desktop/AI-GPT-2/gpt-2/src/<a href="http://model.py/">model.py</a>", line 3, in <module></p> <p> from <a href="http://tensorflow.contrib.training/">tensorflow.contrib.training</a> import HParams</p> <p>ModuleNotFoundError: No module named 'tensorflow.contrib'</p> <p>(base) Dodjis-MacBook-Pro:gpt-2 dodjicohovi$</p> </div> </article> </li> <li id="comment-179785" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-179785" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCv9nZN1Nqq33-O6k4kOg0XQ" class="url" rel="ugc external nofollow">Peter Engelhardt</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179785"><time datetime="2020-07-25T07:36:03-07:00">July 25, 2020 at 7:36 am</time></a> </div> </footer> <div class="comment-content"> <p>I enjoyed your detailed instructions etc now I am just curious to know when you are going to give instructions for GPT-3</p> </div> </article> </li> <li id="comment-179784" class="comment even thread-even depth-1"> <article id="div-comment-179784" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCXdmLFeGX6XFI4dWC94So4g" class="url" rel="ugc external nofollow">assuke</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179784"><time datetime="2020-07-25T12:51:40-07:00">July 25, 2020 at 12:51 pm</time></a> </div> </footer> <div class="comment-content"> <p>lol sudo ! you don't get it dude ..</p> </div> </article> </li> <li id="comment-179783" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-179783" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCGROyJkkJwntJ9LB7tPYcGA" class="url" rel="ugc external nofollow">MJ</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179783"><time datetime="2020-07-28T07:59:20-07:00">July 28, 2020 at 7:59 am</time></a> </div> </footer> <div class="comment-content"> <p>is this tutorial for windows 10 or another operativ system? if this is for beginners i should not look at the pro lessons 🙂</p> </div> </article> </li> <li id="comment-179782" class="comment even thread-even depth-1"> <article id="div-comment-179782" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCsSh_YZrIWoRDWNa0Yjjxaw" class="url" rel="ugc external nofollow">mengjun wang</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179782"><time datetime="2020-08-11T18:29:07-07:00">August 11, 2020 at 6:29 pm</time></a> </div> </footer> <div class="comment-content"> <p>Hi Can I train my own model using 1558M or 775m data instead of using 117M data? and if it is possible, how can Ido that? thank u!</p> </div> </article> </li> <li id="comment-179781" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-179781" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCk8N5oHcNeMIZgQcgMhyTTw" class="url" rel="ugc external nofollow">TheLinuxCult</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/open-ai-gpt-2-beginners-tutorial/#comment-179781"><time datetime="2020-09-22T04:20:25-07:00">September 22, 2020 at 4:20 am</time></a> </div> </footer> <div class="comment-content"> <p>how do I install tensorflow?</p> </div> </article> </li> </ol> <p class="no-comments">Comments are closed.</p> </div> </main> </div> <div class="col-md-3 px-lg-3 "> </div> </div> </div> </div> <footer id="colophon" class="site-footer"> <div class="container"> <div class="row"> <div class="col-md-12 text-center"> <div class="site-info"> <span> Powered By: <a href="https://wordpress.org/" target="_blank">WordPress</a> </span> <span class="sep"> | </span> <span> Theme: <a href="https://odiethemes.com/themes/magazinebook/" target="_blank">MagazineBook</a> By OdieThemes </span> </div> </div> </div> </div> </footer> </div> <script>(function(){var advanced_ads_ga_UID="UA-88163215-1",advanced_ads_ga_anonymIP=!!1;window.advanced_ads_check_adblocker=function(t){var n=[],e=null;function a(t){var n=window.requestAnimationFrame||window.mozRequestAnimationFrame||window.webkitRequestAnimationFrame||function(t){return setTimeout(t,16)};n.call(window,t)}return a((function(){var t=document.createElement("div");t.innerHTML=" ",t.setAttribute("class","ad_unit ad-unit text-ad text_ad pub_300x250"),t.setAttribute("style","width: 1px !important; height: 1px !important; position: absolute !important; left: 0px !important; top: 0px !important; overflow: hidden !important;"),document.body.appendChild(t),a((function(){var a,o,i=null===(a=(o=window).getComputedStyle)||void 0===a?void 0:a.call(o,t),d=null==i?void 0:i.getPropertyValue("-moz-binding");e=i&&"none"===i.getPropertyValue("display")||"string"==typeof d&&-1!==d.indexOf("about:");for(var c=0,r=n.length;c<r;c++)n[c](e);n=[]}))})),function(t){"undefined"==typeof advanced_ads_adblocker_test&&(e=!0),null!==e?t(e):n.push(t)}}(),(()=>{function t(t){this.UID=t,this.analyticsObject="function"==typeof gtag;var n=this;return this.count=function(){gtag("event","AdBlock",{event_category:"Advanced Ads",event_label:"Yes",non_interaction:!0,send_to:n.UID})},function(){if(!n.analyticsObject){var e=document.createElement("script");e.src="https://www.googletagmanager.com/gtag/js?id="+t,e.async=!0,document.body.appendChild(e),window.dataLayer=window.dataLayer||[],window.gtag=function(){dataLayer.push(arguments)},n.analyticsObject=!0,gtag("js",new Date)}var a={send_page_view:!1,transport_type:"beacon"};window.advanced_ads_ga_anonymIP&&(a.anonymize_ip=!0),gtag("config",t,a)}(),this}advanced_ads_check_adblocker((function(n){n&&new t(advanced_ads_ga_UID).count()}))})();})();</script><div style="clear:both;width:100%;text-align:center; font-size:11px; "><a target="_blank" title="WP2Social Auto Publish" href="https://xyzscripts.com/wordpress-plugins/facebook-auto-publish/compare" >WP2Social Auto Publish</a> Powered By : <a target="_blank" title="PHP Scripts & Programs" href="http://www.xyzscripts.com" >XYZScripts.com</a></div><script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/navigation.js?ver=1.0.9" id="magazinebook-navigation-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/skip-link-focus-fix.js?ver=1.0.9" id="magazinebook-skip-link-focus-fix-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/jquery.easy-ticker.js?ver=3.1.0" id="magazinebook-news-ticker-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/splide.min.js?ver=2.3.1" id="splide-js-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/theme.js?ver=1.0.9" id="magazinebook-theme-js-js"></script> <script>!function(){window.advanced_ads_ready_queue=window.advanced_ads_ready_queue||[],advanced_ads_ready_queue.push=window.advanced_ads_ready;for(var d=0,a=advanced_ads_ready_queue.length;d<a;d++)advanced_ads_ready(advanced_ads_ready_queue[d])}();</script> </body> </html>