(Very) Few Shot NLP Tokenization with GPT-3 (even escapes strings)

Bakz T. Future

Could GPT-3 be used for basic NLP tasks like tokenization? I made a video to find out!

—
► Remember to Like, Comment, and Subscribe!
—
Connect with me:
Twitter – http://www.twitter.com/bakztfuture
Instagram – http://www.instagram.com/bakztfuture
Github – http://www.github.com/bakztfuture

Feel free to send me an email or just say hello:
bakztfuture@gmail.com

edited by: Nino Bedoshvili

<iframe></p> <p><a href="https://www.youtube.com/watch?v=4GrLx5XUy9M">Source</a></p> <div class="be1e40beae42d993bafb8643f4ddde8b" data-index="3" style="float: none; margin:10px 0 10px 0; text-align:center;"> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-9244112244416304" data-ad-slot="4549240677"></ins> <script> (adsbygoogle = window.adsbygoogle || []).push({}); </script> </div> <div style="font-size: 0px; height: 0px; line-height: 0px; margin: 0; padding: 0; clear: both;"></div> </div> </article> <div class="clearfix"></div> <ul class="default-theme-post-navigation"> <li class="theme-nav-previous"><a href="https://theengineeringofconsciousexperience.com/the-connected-universe-nassim-haramein-tedxucsd/" rel="prev"><span class="meta-nav">←</span> The Connected Universe | Nassim Haramein | TEDxUCSD</a></li> <li class="theme-nav-next"><a href="https://theengineeringofconsciousexperience.com/deploying-ml-models-gpt-3-monthly-meetup-july-2020/" rel="next">Deploying ML models & GPT-3 | Monthly Meetup – July 2020 <span class="meta-nav">→</span></a></li> </ul> <div class="clearfix"></div> <h3 class='comment-reply-title'>Similar Posts</h3> <div class="mb-related-posts mb-simple-featured-posts mb-simple-featured-posts-wrap row"> <article class="mb-featured-article col-md-4 px-lg-3 post"> <a class="post-thumbnail" href="https://theengineeringofconsciousexperience.com/gpt-3-paper-language-models-are-few-shot-learners-in-depth-presentation/" aria-hidden="true" tabindex="-1"> <img width="501" height="282" src="https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/06/1624210718_maxresdefault.jpg" class="attachment-magazinebook-featured-image-medium size-magazinebook-featured-image-medium wp-post-image" alt="" decoding="async" loading="lazy" srcset="https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/06/1624210718_maxresdefault.jpg 1280w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/06/1624210718_maxresdefault-300x169.jpg 300w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/06/1624210718_maxresdefault-1024x576.jpg 1024w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/06/1624210718_maxresdefault-768x432.jpg 768w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/06/1624210718_maxresdefault-520x293.jpg 520w" sizes="(max-width: 501px) 100vw, 501px" /> </a> <span class="cat-links"><a href="https://theengineeringofconsciousexperience.com/category/gpt-3/" rel="category tag">GPT 3</a></span> <header class="entry-header"> <h3 class="entry-title"><a href="https://theengineeringofconsciousexperience.com/gpt-3-paper-language-models-are-few-shot-learners-in-depth-presentation/" rel="bookmark">GPT-3 Paper: Language Models are Few-Shot Learners – In-Depth Presentation</a></h3> <div class="entry-meta"> <span class="posted-on"><i class="far fa-calendar-alt"></i><a href="https://theengineeringofconsciousexperience.com/gpt-3-paper-language-models-are-few-shot-learners-in-depth-presentation/" rel="bookmark"><time class="entry-date published updated" datetime="2021-05-24T14:27:29-07:00">May 24, 2021</time></a></span><span class="byline"><i class="far fa-user-circle"></i><span class="author vcard"><a class="url fn n" href="https://theengineeringofconsciousexperience.com/author/johannes_hagemann/">Johannes Hagemann</a></span></span> </div> </header> </article> <article class="mb-featured-article col-md-4 px-lg-3 post"> <a class="post-thumbnail" href="https://theengineeringofconsciousexperience.com/openais-sarcastic-chat-bot-gpt-3-api-beta/" aria-hidden="true" tabindex="-1"> <img width="501" height="282" src="https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/05/1620516306_maxresdefault.jpg" class="attachment-magazinebook-featured-image-medium size-magazinebook-featured-image-medium wp-post-image" alt="" decoding="async" loading="lazy" srcset="https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/05/1620516306_maxresdefault.jpg 1280w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/05/1620516306_maxresdefault-300x169.jpg 300w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/05/1620516306_maxresdefault-1024x576.jpg 1024w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/05/1620516306_maxresdefault-768x432.jpg 768w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/05/1620516306_maxresdefault-520x293.jpg 520w" sizes="(max-width: 501px) 100vw, 501px" /> </a> <span class="cat-links"><a href="https://theengineeringofconsciousexperience.com/category/gpt-3/" rel="category tag">GPT 3</a></span> <header class="entry-header"> <h3 class="entry-title"><a href="https://theengineeringofconsciousexperience.com/openais-sarcastic-chat-bot-gpt-3-api-beta/" rel="bookmark">OpenAI’s Sarcastic Chat Bot [GPT-3 API Beta]</a></h3> <div class="entry-meta"> <span class="posted-on"><i class="far fa-calendar-alt"></i><a href="https://theengineeringofconsciousexperience.com/openais-sarcastic-chat-bot-gpt-3-api-beta/" rel="bookmark"><time class="entry-date published updated" datetime="2021-04-05T05:21:21-07:00">April 5, 2021</time></a></span><span class="byline"><i class="far fa-user-circle"></i><span class="author vcard"><a class="url fn n" href="https://theengineeringofconsciousexperience.com/author/f0339c0fb2994df2032cd679f8e25c7a/">bycloud</a></span></span> </div> </header> </article> <article class="mb-featured-article col-md-4 px-lg-3 post"> <a class="post-thumbnail" href="https://theengineeringofconsciousexperience.com/closers-copy-lifetime-deal-purchase-website-review-demonstration-gpt3-openai/" aria-hidden="true" tabindex="-1"> <img width="501" height="282" src="https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/07/1625178897_maxresdefault.jpg" class="attachment-magazinebook-featured-image-medium size-magazinebook-featured-image-medium wp-post-image" alt="" decoding="async" loading="lazy" srcset="https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/07/1625178897_maxresdefault.jpg 1280w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/07/1625178897_maxresdefault-300x169.jpg 300w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/07/1625178897_maxresdefault-1024x576.jpg 1024w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/07/1625178897_maxresdefault-768x432.jpg 768w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/07/1625178897_maxresdefault-520x293.jpg 520w" sizes="(max-width: 501px) 100vw, 501px" /> </a> <span class="cat-links"><a href="https://theengineeringofconsciousexperience.com/category/gpt-3/" rel="category tag">GPT 3</a></span> <header class="entry-header"> <h3 class="entry-title"><a href="https://theengineeringofconsciousexperience.com/closers-copy-lifetime-deal-purchase-website-review-demonstration-gpt3-openai/" rel="bookmark">Closers Copy Lifetime deal purchase website Review, Demonstration – GPT3 OpenAI</a></h3> <div class="entry-meta"> <span class="posted-on"><i class="far fa-calendar-alt"></i><a href="https://theengineeringofconsciousexperience.com/closers-copy-lifetime-deal-purchase-website-review-demonstration-gpt3-openai/" rel="bookmark"><time class="entry-date published updated" datetime="2021-06-27T00:30:54-07:00">June 27, 2021</time></a></span><span class="byline"><i class="far fa-user-circle"></i><span class="author vcard"><a class="url fn n" href="https://theengineeringofconsciousexperience.com/author/vox_eroz/">Vox Eroz</a></span></span> </div> </header> </article> </div> <div id="comments" class="comments-area"> <h5 class="comments-title"> 11 thoughts on “<span>(Very) Few Shot NLP Tokenization with GPT-3 (even escapes strings)</span>” </h5> <ol class="comment-list"> <li id="comment-173908" class="comment even thread-even depth-1"> <article id="div-comment-173908" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCz0clDQsfxFhz_xlmEdQWFw" class="url" rel="ugc external nofollow">Bakz T. Future</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/very-few-shot-nlp-tokenization-with-gpt-3-even-escapes-strings/#comment-173908"><time datetime="2020-07-25T11:24:13-07:00">July 25, 2020 at 11:24 am</time></a> </div> </footer> <div class="comment-content"> <p>Now I'm curious how GPT would act if it was trained on tokenized sentences instead of characters …</p> </div> </article> </li> <li id="comment-173907" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-173907" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCgIHIevRIVZ2eeosDHiDJmg" class="url" rel="ugc external nofollow">Felix Johnson</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/very-few-shot-nlp-tokenization-with-gpt-3-even-escapes-strings/#comment-173907"><time datetime="2020-07-25T11:30:38-07:00">July 25, 2020 at 11:30 am</time></a> </div> </footer> <div class="comment-content"> <p>Hello! Thank you so much for your videos, I enjoy them thoroughly. I'm still just sitting here waiting for my GPT-3 API key, was hoping to ask you some questions. <br />How long does an actual GPT-3 API call take? And how much does it cost right now, per call or per month. </p> <p> Ask GPT-3 if you don't know the answer.</p> </div> </article> </li> <li id="comment-173906" class="comment even thread-even depth-1"> <article id="div-comment-173906" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCvdypSL9asPlTikh6OmpH8w" class="url" rel="ugc external nofollow">Tyfoods For Thought</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/very-few-shot-nlp-tokenization-with-gpt-3-even-escapes-strings/#comment-173906"><time datetime="2020-08-01T08:47:19-07:00">August 1, 2020 at 8:47 am</time></a> </div> </footer> <div class="comment-content"> <p>OoOoOOo neat! Thanks for sharing!</p> </div> </article> </li> <li id="comment-173905" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-173905" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC9z_s_R3DCBTx4N8XCRej3A" class="url" rel="ugc external nofollow">Sugar Papi</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/very-few-shot-nlp-tokenization-with-gpt-3-even-escapes-strings/#comment-173905"><time datetime="2020-08-08T22:15:28-07:00">August 8, 2020 at 10:15 pm</time></a> </div> </footer> <div class="comment-content"> <p>This is crazy</p> </div> </article> </li> <li id="comment-173904" class="comment even thread-even depth-1"> <article id="div-comment-173904" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCFskCaYf4Zd-Km6_KP167cA" class="url" rel="ugc external nofollow">Evan Bialo</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/very-few-shot-nlp-tokenization-with-gpt-3-even-escapes-strings/#comment-173904"><time datetime="2020-08-09T06:33:31-07:00">August 9, 2020 at 6:33 am</time></a> </div> </footer> <div class="comment-content"> <p>Holy shit! GPT-3 is almost a general purpose AI! I'm not even joking when I say a theoretical GPT-4 or 5 could be human level intelligence.</p> </div> </article> </li> <li id="comment-173903" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-173903" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCK3bzdKwoawaVO01d8_zpfQ" class="url" rel="ugc external nofollow">Kenny CanHe</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/very-few-shot-nlp-tokenization-with-gpt-3-even-escapes-strings/#comment-173903"><time datetime="2020-08-10T03:47:28-07:00">August 10, 2020 at 3:47 am</time></a> </div> </footer> <div class="comment-content"> <p>What program are you using for this presentation? ie. <a href="https://www.youtube.com/watch?v=4GrLx5XUy9M&t=0m22s">0:22</a>, and onward from <a href="https://www.youtube.com/watch?v=4GrLx5XUy9M&t=5m33s">5:33</a></p> </div> </article> </li> <li id="comment-173902" class="comment even thread-even depth-1"> <article id="div-comment-173902" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCVbT3bYDry6xmGb2we_fYiw" class="url" rel="ugc external nofollow">dcrey7</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/very-few-shot-nlp-tokenization-with-gpt-3-even-escapes-strings/#comment-173902"><time datetime="2020-08-11T16:59:12-07:00">August 11, 2020 at 4:59 pm</time></a> </div> </footer> <div class="comment-content"> <p>woa this is cool</p> </div> </article> </li> <li id="comment-173901" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-173901" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC0jZ0tR67BRoFo-2Veqm3EA" class="url" rel="ugc external nofollow">Łukasz Rysiak</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/very-few-shot-nlp-tokenization-with-gpt-3-even-escapes-strings/#comment-173901"><time datetime="2020-08-13T22:37:23-07:00">August 13, 2020 at 10:37 pm</time></a> </div> </footer> <div class="comment-content"> <p><a href="https://www.youtube.com/watch?v=4GrLx5XUy9M&t=5m40s">5:40</a> what? You're basically using an API connected to model trained on terabytes of data and you're saying that this is crazy that you don't have to download libraries and datasets?</p> </div> </article> </li> <li id="comment-173900" class="comment even thread-even depth-1"> <article id="div-comment-173900" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC2FvwVZKzdHIPNdt4yaT8Qw" class="url" rel="ugc external nofollow">Ralph Dratman</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/very-few-shot-nlp-tokenization-with-gpt-3-even-escapes-strings/#comment-173900"><time datetime="2020-08-18T17:57:48-07:00">August 18, 2020 at 5:57 pm</time></a> </div> </footer> <div class="comment-content"> <p>What would happen if you gave GPT-3 zero examples and just asked it to tokenize? <br />Maybe it already knows enough to do some kind of tokenization without any hints.</p> <p>I do not think GPT-3 "understands" anything. It is just mechanically acting the way it was created to act.</p> </div> </article> </li> <li id="comment-173899" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-173899" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC1usKk-H-_kUlk1FYbX6SwA" class="url" rel="ugc external nofollow">Rasmus Schultz</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/very-few-shot-nlp-tokenization-with-gpt-3-even-escapes-strings/#comment-173899"><time datetime="2020-09-01T22:44:16-07:00">September 1, 2020 at 10:44 pm</time></a> </div> </footer> <div class="comment-content"> <p>One issue not addressed here is reliability. It won't always return the same result for the same query – there's a randomness factor, which makes it largely unacceptable for cases like this. Besides the obvious, the performance issue – which, no, optimizations won't solve that, not by a long shot. That said, ask it to generate code or regular expressions (both have been done) and you might get something reliable and performant that you could unit test.</p> </div> </article> </li> <li id="comment-173898" class="comment even thread-even depth-1"> <article id="div-comment-173898" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCqREGS9X-axV45jVPMAeyEw" class="url" rel="ugc external nofollow">Simon Janin</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/very-few-shot-nlp-tokenization-with-gpt-3-even-escapes-strings/#comment-173898"><time datetime="2020-09-04T10:37:53-07:00">September 4, 2020 at 10:37 am</time></a> </div> </footer> <div class="comment-content"> <p>This is excellent. <br />One caveat: no way GPT could be faster than a specialized algorithm, ever. It's probably not what you meant at the end of the video</p> </div> </article> </li> </ol> <p class="no-comments">Comments are closed.</p> </div> </main> </div> <div class="col-md-3 px-lg-3 "> </div> </div> </div> </div> <footer id="colophon" class="site-footer"> <div class="container"> <div class="row"> <div class="col-md-12 text-center"> <div class="site-info"> <span> Powered By: <a href="https://wordpress.org/" target="_blank">WordPress</a> </span> <span class="sep"> | </span> <span> Theme: <a href="https://odiethemes.com/themes/magazinebook/" target="_blank">MagazineBook</a> By OdieThemes </span> </div> </div> </div> </div> </footer> </div> <script>(function(){var advanced_ads_ga_UID="UA-88163215-1",advanced_ads_ga_anonymIP=!!1;window.advanced_ads_check_adblocker=function(t){var n=[],e=null;function a(t){var n=window.requestAnimationFrame||window.mozRequestAnimationFrame||window.webkitRequestAnimationFrame||function(t){return setTimeout(t,16)};n.call(window,t)}return a((function(){var t=document.createElement("div");t.innerHTML=" ",t.setAttribute("class","ad_unit ad-unit text-ad text_ad pub_300x250"),t.setAttribute("style","width: 1px !important; height: 1px !important; position: absolute !important; left: 0px !important; top: 0px !important; overflow: hidden !important;"),document.body.appendChild(t),a((function(){var a,o,i=null===(a=(o=window).getComputedStyle)||void 0===a?void 0:a.call(o,t),d=null==i?void 0:i.getPropertyValue("-moz-binding");e=i&&"none"===i.getPropertyValue("display")||"string"==typeof d&&-1!==d.indexOf("about:");for(var c=0,r=n.length;c<r;c++)n[c](e);n=[]}))})),function(t){"undefined"==typeof advanced_ads_adblocker_test&&(e=!0),null!==e?t(e):n.push(t)}}(),function(){function t(t){this.UID=t,this.analyticsObject="function"==typeof gtag;var n=this;return this.count=function(){gtag("event","AdBlock",{event_category:"Advanced Ads",event_label:"Yes",non_interaction:!0,send_to:n.UID})},function(){if(!n.analyticsObject){var e=document.createElement("script");e.src="https://www.googletagmanager.com/gtag/js?id="+t,e.async=!0,document.body.appendChild(e),window.dataLayer=window.dataLayer||[],window.gtag=function(){dataLayer.push(arguments)},n.analyticsObject=!0,gtag("js",new Date)}var a={send_page_view:!1,transport_type:"beacon"};window.advanced_ads_ga_anonymIP&&(a.anonymize_ip=!0),gtag("config",t,a)}(),this}advanced_ads_check_adblocker((function(n){n&&new t(advanced_ads_ga_UID).count()}))}();})();</script><div style="clear:both;width:100%;text-align:center; font-size:11px; "><a target="_blank" title="WP2Social Auto Publish" href="https://xyzscripts.com/wordpress-plugins/facebook-auto-publish/compare" >WP2Social Auto Publish</a> Powered By : <a target="_blank" title="PHP Scripts & Programs" href="http://www.xyzscripts.com" >XYZScripts.com</a></div><script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/navigation.js?ver=1.0.9" id="magazinebook-navigation-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/skip-link-focus-fix.js?ver=1.0.9" id="magazinebook-skip-link-focus-fix-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/jquery.easy-ticker.js?ver=3.1.0" id="magazinebook-news-ticker-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/splide.min.js?ver=2.3.1" id="splide-js-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/theme.js?ver=1.0.9" id="magazinebook-theme-js-js"></script> <script>!function(){window.advanced_ads_ready_queue=window.advanced_ads_ready_queue||[],advanced_ads_ready_queue.push=window.advanced_ads_ready;for(var d=0,a=advanced_ads_ready_queue.length;d<a;d++)advanced_ads_ready(advanced_ads_ready_queue[d])}();</script> </body> </html>