GPT-3: A Quick Overview – The Engineering of Conscious Experience

Applied AI Course

Notes: https://drive.google.com/file/d/1hY2y1U2vJ94QCgGw2HjJ2-0Eo9ogA7T6/view?usp=sharing

<iframe></p> <p><a href="https://www.youtube.com/watch?v=_xBvvwrAOxk">Source</a></p> <div class="be1e40beae42d993bafb8643f4ddde8b" data-index="3" style="float: none; margin:10px 0 10px 0; text-align:center;"> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-9244112244416304" data-ad-slot="4549240677"></ins> <script> (adsbygoogle = window.adsbygoogle || []).push({}); </script> </div> <div style="font-size: 0px; height: 0px; line-height: 0px; margin: 0; padding: 0; clear: both;"></div> </div> </article> <div class="clearfix"></div> <ul class="default-theme-post-navigation"> <li class="theme-nav-previous"><a href="https://theengineeringofconsciousexperience.com/agree-meaning-agree-meaning-in-hindi-meaning-of-agree-in-urdu/" rel="prev"><span class="meta-nav">←</span> Agree meaning | Agree Meaning In Hindi | Meaning of Agree in Urdu</a></li> <li class="theme-nav-next"><a href="https://theengineeringofconsciousexperience.com/meditacion-guiada-mindfulness-para-cultivar-la-gratitud-mindful-science/" rel="next">Meditación Guiada Mindfulness para cultivar la GRATITUD | Mindful Science <span class="meta-nav">→</span></a></li> </ul> <div class="clearfix"></div> <h3 class='comment-reply-title'>Similar Posts</h3> <div class="mb-related-posts mb-simple-featured-posts mb-simple-featured-posts-wrap row"> <article class="mb-featured-article col-md-4 px-lg-3 post"> <a class="post-thumbnail" href="https://theengineeringofconsciousexperience.com/gpt-4-vision-10-amazing-use-cases-this-is-huge/" aria-hidden="true" tabindex="-1"> <img width="501" height="300" src="https://theengineeringofconsciousexperience.com/wp-content/uploads/2023/11/1698978383_maxresdefault-501x300.jpg" class="attachment-magazinebook-featured-image-medium size-magazinebook-featured-image-medium wp-post-image" alt="" decoding="async" loading="lazy" /> </a> <span class="cat-links"><a href="https://theengineeringofconsciousexperience.com/category/gpt-3/" rel="category tag">GPT 3</a></span> <header class="entry-header"> <h3 class="entry-title"><a href="https://theengineeringofconsciousexperience.com/gpt-4-vision-10-amazing-use-cases-this-is-huge/" rel="bookmark">GPT-4 Vision: 10 Amazing Use Cases – This is HUGE!!</a></h3> <div class="entry-meta"> <span class="posted-on"><i class="far fa-calendar-alt"></i><a href="https://theengineeringofconsciousexperience.com/gpt-4-vision-10-amazing-use-cases-this-is-huge/" rel="bookmark"><time class="entry-date published updated" datetime="2023-10-13T07:28:44-07:00">October 13, 2023</time></a></span><span class="byline"><i class="far fa-user-circle"></i><span class="author vcard"><a class="url fn n" href="https://theengineeringofconsciousexperience.com/author/all_about_ai/">All About AI</a></span></span> </div> </header> </article> <article class="mb-featured-article col-md-4 px-lg-3 post"> <a class="post-thumbnail" href="https://theengineeringofconsciousexperience.com/stream-southbird-playing-with-dragons-part-3-starbound-futaras-dragon-race/" aria-hidden="true" tabindex="-1"> <img width="501" height="282" src="https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/12/1608677276_maxresdefault.jpg" class="attachment-magazinebook-featured-image-medium size-magazinebook-featured-image-medium wp-post-image" alt="" decoding="async" loading="lazy" srcset="https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/12/1608677276_maxresdefault.jpg 1280w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/12/1608677276_maxresdefault-300x169.jpg 300w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/12/1608677276_maxresdefault-1024x576.jpg 1024w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/12/1608677276_maxresdefault-768x432.jpg 768w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/12/1608677276_maxresdefault-520x293.jpg 520w" sizes="(max-width: 501px) 100vw, 501px" /> </a> <span class="cat-links"><a href="https://theengineeringofconsciousexperience.com/category/gpt-3/" rel="category tag">GPT 3</a></span> <header class="entry-header"> <h3 class="entry-title"><a href="https://theengineeringofconsciousexperience.com/stream-southbird-playing-with-dragons-part-3-starbound-futaras-dragon-race/" rel="bookmark">STREAM: Southbird Playing with Dragons Part 3 (Starbound, Futara's Dragon Race)</a></h3> <div class="entry-meta"> <span class="posted-on"><i class="far fa-calendar-alt"></i><a href="https://theengineeringofconsciousexperience.com/stream-southbird-playing-with-dragons-part-3-starbound-futaras-dragon-race/" rel="bookmark"><time class="entry-date published updated" datetime="2020-12-12T18:19:46-07:00">December 12, 2020</time></a></span><span class="byline"><i class="far fa-user-circle"></i><span class="author vcard"><a class="url fn n" href="https://theengineeringofconsciousexperience.com/author/b9d801ead21b27ca99b526c5c674126f/">Captain Southbird</a></span></span> </div> </header> </article> <article class="mb-featured-article col-md-4 px-lg-3 post"> <a class="post-thumbnail" href="https://theengineeringofconsciousexperience.com/gpt-3-can-you-tell-the-difference-between-a-human-and-gpt-3/" aria-hidden="true" tabindex="-1"> <img width="501" height="282" src="https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/10/1602808053_maxresdefault.jpg" class="attachment-magazinebook-featured-image-medium size-magazinebook-featured-image-medium wp-post-image" alt="" decoding="async" loading="lazy" srcset="https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/10/1602808053_maxresdefault.jpg 1280w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/10/1602808053_maxresdefault-300x169.jpg 300w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/10/1602808053_maxresdefault-1024x576.jpg 1024w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/10/1602808053_maxresdefault-768x432.jpg 768w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/10/1602808053_maxresdefault-520x293.jpg 520w" sizes="(max-width: 501px) 100vw, 501px" /> </a> <span class="cat-links"><a href="https://theengineeringofconsciousexperience.com/category/gpt-3/" rel="category tag">GPT 3</a></span> <header class="entry-header"> <h3 class="entry-title"><a href="https://theengineeringofconsciousexperience.com/gpt-3-can-you-tell-the-difference-between-a-human-and-gpt-3/" rel="bookmark">GPT-3 – Can you tell the difference between a human and GPT-3?</a></h3> <div class="entry-meta"> <span class="posted-on"><i class="far fa-calendar-alt"></i><a href="https://theengineeringofconsciousexperience.com/gpt-3-can-you-tell-the-difference-between-a-human-and-gpt-3/" rel="bookmark"><time class="entry-date published updated" datetime="2020-08-25T12:38:34-07:00">August 25, 2020</time></a></span><span class="byline"><i class="far fa-user-circle"></i><span class="author vcard"><a class="url fn n" href="https://theengineeringofconsciousexperience.com/author/71855dd795300e2efd24bb5af581dd02/">FoxCrow</a></span></span> </div> </header> </article> </div> <div id="comments" class="comments-area"> <h5 class="comments-title"> 12 thoughts on “<span>GPT-3: A Quick Overview</span>” </h5> <ol class="comment-list"> <li id="comment-168809" class="comment even thread-even depth-1"> <article id="div-comment-168809" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UClGXmKQicpqHk_eYQk03sTQ" class="url" rel="ugc external nofollow">Sachin Kalsi</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/gpt-3-a-quick-overview/#comment-168809"><time datetime="2020-08-02T23:14:43-07:00">August 2, 2020 at 11:14 pm</time></a> </div> </footer> <div class="comment-content"> <p>Today's language models ingest far more text than a human could read in a <a href="http://www.youtube.com/results?search_query=%23lifetime">#lifetime</a> reveals both the power of brute-force training and the <a href="http://www.youtube.com/results?search_query=%23algorithms">#algorithms</a> inefficiency at learning – @deeplearningai_</p> <p>What's your comment on this, Sir?</p> </div> </article> </li> <li id="comment-168808" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-168808" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCzkdPPyl4U3P6xJuRYYe7zQ" class="url" rel="ugc external nofollow">saisushanth durvasula</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/gpt-3-a-quick-overview/#comment-168808"><time datetime="2020-08-03T00:08:43-07:00">August 3, 2020 at 12:08 am</time></a> </div> </footer> <div class="comment-content"> <p>Could you please provide good resources for the pre reqs mentioned in the video sir , so that we can understand this live session ?</p> </div> </article> </li> <li id="comment-168807" class="comment even thread-even depth-1"> <article id="div-comment-168807" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCUJXXQRb8utQi9GJ-lDAOqA" class="url" rel="ugc external nofollow">Vineet Samudrala</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/gpt-3-a-quick-overview/#comment-168807"><time datetime="2020-08-03T00:21:13-07:00">August 3, 2020 at 12:21 am</time></a> </div> </footer> <div class="comment-content"> <p>For people wanting to see what the gpt – 3 model is capable of .. I would highly suggest this hackathon winning application made possible with the help of the gpt-3 model. <br /><a href="https://play.aidungeon.io/">https://play.aidungeon.io/</a></p> </div> </article> </li> <li id="comment-168806" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-168806" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCeudgSeiyxB6xVHdm6zjkOg" class="url" rel="ugc external nofollow">janaki raman</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/gpt-3-a-quick-overview/#comment-168806"><time datetime="2020-08-03T00:42:18-07:00">August 3, 2020 at 12:42 am</time></a> </div> </footer> <div class="comment-content"> <p>Is it different from any search engine? Because we train on data from the internet, What's the difference between a search engine's reply and GPT-3 reply in case of "QNA". Please ignore in case if it is a stupid question. I am trying to understand the difference.</p> </div> </article> </li> <li id="comment-168805" class="comment even thread-even depth-1"> <article id="div-comment-168805" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCQCnuWzLMOUQ2DC1Li9I3nQ" class="url" rel="ugc external nofollow">Ambuj Mittal</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/gpt-3-a-quick-overview/#comment-168805"><time datetime="2020-08-03T08:03:24-07:00">August 3, 2020 at 8:03 am</time></a> </div> </footer> <div class="comment-content"> <p>Training GPT-3 cost around 4.2M US Dollars for OpenAi. I think that even though the database size won't be a problem for companies like Google but getting an architecture of this level to train your network on will certainly be a huge challenge, even for them.</p> </div> </article> </li> <li id="comment-168804" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-168804" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCu9eaWbcMZlrEHNjXJAGQuw" class="url" rel="ugc external nofollow">Sourabh Ghorpade</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/gpt-3-a-quick-overview/#comment-168804"><time datetime="2020-08-03T22:13:04-07:00">August 3, 2020 at 10:13 pm</time></a> </div> </footer> <div class="comment-content"> <p>Thanks for the resources,will be prepared for the live sessions now! Looking forward to live sessions.</p> </div> </article> </li> <li id="comment-168803" class="comment even thread-even depth-1"> <article id="div-comment-168803" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCIKCdJ4d3w188DF5WdUTo_Q" class="url" rel="ugc external nofollow">Altaf S</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/gpt-3-a-quick-overview/#comment-168803"><time datetime="2020-08-03T23:34:53-07:00">August 3, 2020 at 11:34 pm</time></a> </div> </footer> <div class="comment-content"> <p>Hi Team, please suggest books for statistics, and ML.<br />Also is “oreilly hands on ML with scikit” a good book to go ?</p> </div> </article> </li> <li id="comment-168802" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-168802" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCxntxRIY4vwIPBW-b-j06tg" class="url" rel="ugc external nofollow">Devansh Khandekar</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/gpt-3-a-quick-overview/#comment-168802"><time datetime="2020-08-04T00:30:31-07:00">August 4, 2020 at 12:30 am</time></a> </div> </footer> <div class="comment-content"> <p>As per My knowledge, Turing NLG had the highest number of parameters before GPT3 and it didn't get enough attention . Not just the parameters are a matter of high compute but the FLOPS involved in the rigorous decoder in GPT3 are in the order of 10^23 which makes it very computationally infeasible for anyone. I haven't heard of pruning on GPT and other NLG like models. This suggests in the coming future you might expect SOTA NLG models only with higher parameters and beyond the reach of individual researchers. The ongoing hype is just favoring this development of AI (and not democratizing AI without black boxes). Exciting times ahead but with very less control over it.</p> </div> </article> </li> <li id="comment-168801" class="comment even thread-even depth-1"> <article id="div-comment-168801" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCrdwPEtt0B7Hnn2SN9_vzQA" class="url" rel="ugc external nofollow">Arun Kumar</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/gpt-3-a-quick-overview/#comment-168801"><time datetime="2020-08-04T02:15:46-07:00">August 4, 2020 at 2:15 am</time></a> </div> </footer> <div class="comment-content"> <p>In which problem we use gpt 3</p> </div> </article> </li> <li id="comment-168800" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-168800" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCpirkkub0jzLLN1EbOaUbzw" class="url" rel="ugc external nofollow">Atul patel</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/gpt-3-a-quick-overview/#comment-168800"><time datetime="2020-08-04T11:30:22-07:00">August 4, 2020 at 11:30 am</time></a> </div> </footer> <div class="comment-content"> <p>What I like most the way you explain the concepts in a simple manner. I really like your teaching style.</p> </div> </article> </li> <li id="comment-168799" class="comment even thread-even depth-1"> <article id="div-comment-168799" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC6tBT5jQSX5fWsisqDcv6Tg" class="url" rel="ugc external nofollow">Shrey Mishra</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/gpt-3-a-quick-overview/#comment-168799"><time datetime="2020-08-13T02:18:33-07:00">August 13, 2020 at 2:18 am</time></a> </div> </footer> <div class="comment-content"> <p>As far as I know the GPT-3 model requires API key access which is not easy to get ? am I correct ?</p> </div> </article> </li> <li id="comment-168798" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-168798" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCIQMq23u6QQb6uMUhMwy5PQ" class="url" rel="ugc external nofollow">anand pandiyan</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/gpt-3-a-quick-overview/#comment-168798"><time datetime="2020-08-18T01:06:33-07:00">August 18, 2020 at 1:06 am</time></a> </div> </footer> <div class="comment-content"> <p>Why BERT is having only encoder and gpt have only decoder ?</p> </div> </article> </li> </ol> <p class="no-comments">Comments are closed.</p> </div> </main> </div> <div class="col-md-3 px-lg-3 "> </div> </div> </div> </div> <footer id="colophon" class="site-footer"> <div class="container"> <div class="row"> <div class="col-md-12 text-center"> <div class="site-info"> <span> Powered By: <a href="https://wordpress.org/" target="_blank">WordPress</a> </span> <span class="sep"> | </span> <span> Theme: <a href="https://odiethemes.com/themes/magazinebook/" target="_blank">MagazineBook</a> By OdieThemes </span> </div> </div> </div> </div> </footer> </div> <script>(function(){var advanced_ads_ga_UID="UA-88163215-1",advanced_ads_ga_anonymIP=!!1;window.advanced_ads_check_adblocker=function(t){var n=[],e=null;function a(t){var n=window.requestAnimationFrame||window.mozRequestAnimationFrame||window.webkitRequestAnimationFrame||function(t){return setTimeout(t,16)};n.call(window,t)}return a((function(){var t=document.createElement("div");t.innerHTML=" ",t.setAttribute("class","ad_unit ad-unit text-ad text_ad pub_300x250"),t.setAttribute("style","width: 1px !important; height: 1px !important; position: absolute !important; left: 0px !important; top: 0px !important; overflow: hidden !important;"),document.body.appendChild(t),a((function(){var a,o,i=null===(a=(o=window).getComputedStyle)||void 0===a?void 0:a.call(o,t),d=null==i?void 0:i.getPropertyValue("-moz-binding");e=i&&"none"===i.getPropertyValue("display")||"string"==typeof d&&-1!==d.indexOf("about:");for(var c=0,r=n.length;c<r;c++)n[c](e);n=[]}))})),function(t){"undefined"==typeof advanced_ads_adblocker_test&&(e=!0),null!==e?t(e):n.push(t)}}(),(()=>{function t(t){this.UID=t,this.analyticsObject="function"==typeof gtag;var n=this;return this.count=function(){gtag("event","AdBlock",{event_category:"Advanced Ads",event_label:"Yes",non_interaction:!0,send_to:n.UID})},function(){if(!n.analyticsObject){var e=document.createElement("script");e.src="https://www.googletagmanager.com/gtag/js?id="+t,e.async=!0,document.body.appendChild(e),window.dataLayer=window.dataLayer||[],window.gtag=function(){dataLayer.push(arguments)},n.analyticsObject=!0,gtag("js",new Date)}var a={send_page_view:!1,transport_type:"beacon"};window.advanced_ads_ga_anonymIP&&(a.anonymize_ip=!0),gtag("config",t,a)}(),this}advanced_ads_check_adblocker((function(n){n&&new t(advanced_ads_ga_UID).count()}))})();})();</script><div style="clear:both;width:100%;text-align:center; font-size:11px; "><a target="_blank" title="WP2Social Auto Publish" href="https://xyzscripts.com/wordpress-plugins/facebook-auto-publish/compare" >WP2Social Auto Publish</a> Powered By : <a target="_blank" title="PHP Scripts & Programs" href="http://www.xyzscripts.com" >XYZScripts.com</a></div><script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/navigation.js?ver=1.0.9" id="magazinebook-navigation-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/skip-link-focus-fix.js?ver=1.0.9" id="magazinebook-skip-link-focus-fix-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/jquery.easy-ticker.js?ver=3.1.0" id="magazinebook-news-ticker-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/splide.min.js?ver=2.3.1" id="splide-js-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/theme.js?ver=1.0.9" id="magazinebook-theme-js-js"></script> <script>!function(){window.advanced_ads_ready_queue=window.advanced_ads_ready_queue||[],advanced_ads_ready_queue.push=window.advanced_ads_ready;for(var d=0,a=advanced_ads_ready_queue.length;d<a;d++)advanced_ads_ready(advanced_ads_ready_queue[d])}();</script> </body> </html>