BERT vs GPT – The Engineering of Conscious Experience

CodeEmporium

#machinelearning #shorts #deeplearning #chatgpt #neuralnetwork #datascience

<iframe></p> <p><a href="https://www.youtube.com/watch?v=BEt_BACGw6g">Source</a></p> <div class="be1e40beae42d993bafb8643f4ddde8b" data-index="3" style="float: none; margin:10px 0 10px 0; text-align:center;"> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-9244112244416304" data-ad-slot="4549240677"></ins> <script> (adsbygoogle = window.adsbygoogle || []).push({}); </script> </div> <div style="font-size: 0px; height: 0px; line-height: 0px; margin: 0; padding: 0; clear: both;"></div> </div> </article> <div class="clearfix"></div> <ul class="default-theme-post-navigation"> <li class="theme-nav-previous"><a href="https://theengineeringofconsciousexperience.com/longs-logic-sequel-shorts/" rel="prev"><span class="meta-nav">←</span> Long's Logic – Sequel #shorts</a></li> <li class="theme-nav-next"><a href="https://theengineeringofconsciousexperience.com/chatgpt-tools-that-make-you-a-productivity-god/" rel="next">ChatGPT Tools That Make You a PRODUCTIVITY GOD <span class="meta-nav">→</span></a></li> </ul> <div class="clearfix"></div> <h3 class='comment-reply-title'>Similar Posts</h3> <div class="mb-related-posts mb-simple-featured-posts mb-simple-featured-posts-wrap row"> <article class="mb-featured-article col-md-4 px-lg-3 post"> <a class="post-thumbnail" href="https://theengineeringofconsciousexperience.com/steamed-hams-but-everything-is-fully-explained/" aria-hidden="true" tabindex="-1"> <img width="501" height="282" src="https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/05/1622244245_maxresdefault.jpg" class="attachment-magazinebook-featured-image-medium size-magazinebook-featured-image-medium wp-post-image" alt="" decoding="async" loading="lazy" srcset="https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/05/1622244245_maxresdefault.jpg 1280w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/05/1622244245_maxresdefault-300x169.jpg 300w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/05/1622244245_maxresdefault-1024x576.jpg 1024w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/05/1622244245_maxresdefault-768x432.jpg 768w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/05/1622244245_maxresdefault-520x293.jpg 520w" sizes="(max-width: 501px) 100vw, 501px" /> </a> <span class="cat-links"><a href="https://theengineeringofconsciousexperience.com/category/gpt-3/" rel="category tag">GPT 3</a></span> <header class="entry-header"> <h3 class="entry-title"><a href="https://theengineeringofconsciousexperience.com/steamed-hams-but-everything-is-fully-explained/" rel="bookmark">Steamed Hams but everything is fully explained</a></h3> <div class="entry-meta"> <span class="posted-on"><i class="far fa-calendar-alt"></i><a href="https://theengineeringofconsciousexperience.com/steamed-hams-but-everything-is-fully-explained/" rel="bookmark"><time class="entry-date published updated" datetime="2021-05-15T09:28:14-07:00">May 15, 2021</time></a></span><span class="byline"><i class="far fa-user-circle"></i><span class="author vcard"><a class="url fn n" href="https://theengineeringofconsciousexperience.com/author/b251bfe7b682335894fc421483deafb7/">J.J. McCullough</a></span></span> </div> </header> </article> <article class="mb-featured-article col-md-4 px-lg-3 post"> <a class="post-thumbnail" href="https://theengineeringofconsciousexperience.com/4-the-future-of-ai-introduction-to-gpt-3-a-leap-in-artificial-intelligence/" aria-hidden="true" tabindex="-1"> <img width="400" height="300" src="https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/06/1624115585_hqdefault.jpg" class="attachment-magazinebook-featured-image-medium size-magazinebook-featured-image-medium wp-post-image" alt="" decoding="async" loading="lazy" srcset="https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/06/1624115585_hqdefault.jpg 480w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/06/1624115585_hqdefault-300x225.jpg 300w" sizes="(max-width: 400px) 100vw, 400px" /> </a> <span class="cat-links"><a href="https://theengineeringofconsciousexperience.com/category/gpt-3/" rel="category tag">GPT 3</a></span> <header class="entry-header"> <h3 class="entry-title"><a href="https://theengineeringofconsciousexperience.com/4-the-future-of-ai-introduction-to-gpt-3-a-leap-in-artificial-intelligence/" rel="bookmark">4. The Future of AI | Introduction to GPT-3 A Leap in Artificial Intelligence</a></h3> <div class="entry-meta"> <span class="posted-on"><i class="far fa-calendar-alt"></i><a href="https://theengineeringofconsciousexperience.com/4-the-future-of-ai-introduction-to-gpt-3-a-leap-in-artificial-intelligence/" rel="bookmark"><time class="entry-date published updated" datetime="2021-04-28T20:00:21-07:00">April 28, 2021</time></a></span><span class="byline"><i class="far fa-user-circle"></i><span class="author vcard"><a class="url fn n" href="https://theengineeringofconsciousexperience.com/author/tipzia_learning/">Tipzia Learning</a></span></span> </div> </header> </article> <article class="mb-featured-article col-md-4 px-lg-3 post"> <a class="post-thumbnail" href="https://theengineeringofconsciousexperience.com/will-gpt-3-replace-designers-gpt-3-vs-human-brain/" aria-hidden="true" tabindex="-1"> <img width="501" height="282" src="https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/11/1605260639_maxresdefault.jpg" class="attachment-magazinebook-featured-image-medium size-magazinebook-featured-image-medium wp-post-image" alt="" decoding="async" loading="lazy" srcset="https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/11/1605260639_maxresdefault.jpg 1280w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/11/1605260639_maxresdefault-300x169.jpg 300w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/11/1605260639_maxresdefault-1024x576.jpg 1024w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/11/1605260639_maxresdefault-768x432.jpg 768w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/11/1605260639_maxresdefault-520x293.jpg 520w" sizes="(max-width: 501px) 100vw, 501px" /> </a> <span class="cat-links"><a href="https://theengineeringofconsciousexperience.com/category/gpt-3/" rel="category tag">GPT 3</a></span> <header class="entry-header"> <h3 class="entry-title"><a href="https://theengineeringofconsciousexperience.com/will-gpt-3-replace-designers-gpt-3-vs-human-brain/" rel="bookmark">Will GPT 3 replace Designers ? | GPT 3 vs Human Brain</a></h3> <div class="entry-meta"> <span class="posted-on"><i class="far fa-calendar-alt"></i><a href="https://theengineeringofconsciousexperience.com/will-gpt-3-replace-designers-gpt-3-vs-human-brain/" rel="bookmark"><time class="entry-date published updated" datetime="2020-10-23T10:12:59-07:00">October 23, 2020</time></a></span><span class="byline"><i class="far fa-user-circle"></i><span class="author vcard"><a class="url fn n" href="https://theengineeringofconsciousexperience.com/author/3b75a73ec05ecafabf54717853664606/">AnuragFolio</a></span></span> </div> </header> </article> </div> <div id="comments" class="comments-area"> <h5 class="comments-title"> 11 thoughts on “<span>BERT vs GPT</span>” </h5> <ol class="comment-list"> <li id="comment-313104" class="comment even thread-even depth-1"> <article id="div-comment-313104" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC5fwedajKdGMt05Eb9XlscA" class="url" rel="ugc external nofollow">Varun Tulsian</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/bert-vs-gpt/#comment-313104"><time datetime="2023-01-23T07:46:04-07:00">January 23, 2023 at 7:46 am</time></a> </div> </footer> <div class="comment-content"> <p>this is very useful. Just wanted to add that the gpt decoder doesn't have the cross attention in the transformer block.</p> </div> </article> </li> <li id="comment-313103" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-313103" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCm4vvTEI1bG7bvMCjno0Dfw" class="url" rel="ugc external nofollow">Darshan Tank</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/bert-vs-gpt/#comment-313103"><time datetime="2023-02-16T11:50:57-07:00">February 16, 2023 at 11:50 am</time></a> </div> </footer> <div class="comment-content"> <p>One is for Natural language understanding and another is for Natural language generation</p> </div> </article> </li> <li id="comment-313102" class="comment even thread-even depth-1"> <article id="div-comment-313102" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCoIExWgW2zrBarevLvSfBVw" class="url" rel="ugc external nofollow">Mani N</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/bert-vs-gpt/#comment-313102"><time datetime="2023-02-22T08:22:32-07:00">February 22, 2023 at 8:22 am</time></a> </div> </footer> <div class="comment-content"> <p>Great explanation. For eg, if I have to read all the client emails and understand their requirements and auto create tasks based on that prediction, which model should I go for? BERT or GPT?</p> </div> </article> </li> <li id="comment-313101" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-313101" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCUb0WqFydQYf9yY63B9QX4g" class="url" rel="ugc external nofollow">Obieda Ananbeh</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/bert-vs-gpt/#comment-313101"><time datetime="2023-03-02T12:57:41-07:00">March 2, 2023 at 12:57 pm</time></a> </div> </footer> <div class="comment-content"> <p>Awesome 👏</p> </div> </article> </li> <li id="comment-313100" class="comment even thread-even depth-1"> <article id="div-comment-313100" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCBLJy-IzwLY3gkrNQI8yABA" class="url" rel="ugc external nofollow">Usama</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/bert-vs-gpt/#comment-313100"><time datetime="2023-03-22T17:09:01-07:00">March 22, 2023 at 5:09 pm</time></a> </div> </footer> <div class="comment-content"> <p>I love you ❤</p> </div> </article> </li> <li id="comment-313099" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-313099" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCmGv4qrkENcFTAgcOkhKlhA" class="url" rel="ugc external nofollow">Доктор Ларкен</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/bert-vs-gpt/#comment-313099"><time datetime="2023-05-07T07:27:11-07:00">May 7, 2023 at 7:27 am</time></a> </div> </footer> <div class="comment-content"> <p>Bert also Drives a trans am!</p> <p>Jokes aside I do appreciate your videos!</p> </div> </article> </li> <li id="comment-313098" class="comment even thread-even depth-1"> <article id="div-comment-313098" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCHEyOoYKXzHLnYvjWDLzOUQ" class="url" rel="ugc external nofollow">Vladislav Korecký</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/bert-vs-gpt/#comment-313098"><time datetime="2023-06-14T23:50:45-07:00">June 14, 2023 at 11:50 pm</time></a> </div> </footer> <div class="comment-content"> <p>What if I stack both encoders and decoders? Do I get some BERTGPT hybrid?</p> </div> </article> </li> <li id="comment-313097" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-313097" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCgeHiosnGnNy0FB_JbXnsrQ" class="url" rel="ugc external nofollow">hub git</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/bert-vs-gpt/#comment-313097"><time datetime="2023-07-01T20:28:37-07:00">July 1, 2023 at 8:28 pm</time></a> </div> </footer> <div class="comment-content"> <p>good</p> </div> </article> </li> <li id="comment-313096" class="comment even thread-even depth-1"> <article id="div-comment-313096" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCT_DU9rStmfydZyRNw4JYNA" class="url" rel="ugc external nofollow">Nicholas Zustak</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/bert-vs-gpt/#comment-313096"><time datetime="2023-07-29T10:26:09-07:00">July 29, 2023 at 10:26 am</time></a> </div> </footer> <div class="comment-content"> <p>So BERT doesn’t have a decoder? Did I misunderstand</p> </div> </article> </li> <li id="comment-313095" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-313095" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCVglFCFc_NwpFk63f_GN76Q" class="url" rel="ugc external nofollow">sai madhavi yalla</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/bert-vs-gpt/#comment-313095"><time datetime="2023-08-20T23:13:24-07:00">August 20, 2023 at 11:13 pm</time></a> </div> </footer> <div class="comment-content"> <p>transformer models are usually parallelly run right?</p> </div> </article> </li> <li id="comment-313094" class="comment even thread-even depth-1"> <article id="div-comment-313094" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCAfacq--T8J4pq5lrSsGVRQ" class="url" rel="ugc external nofollow">Vadim.B.Smirnov@phystech AntLion cocolacre</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/bert-vs-gpt/#comment-313094"><time datetime="2023-09-10T01:13:52-07:00">September 10, 2023 at 1:13 am</time></a> </div> </footer> <div class="comment-content"> <p>can't read this scheme, unfathomable. arrow goes inwards to both inputs and outputs? looks like horseshit to me.</p> </div> </article> </li> </ol> <p class="no-comments">Comments are closed.</p> </div> </main> </div> <div class="col-md-3 px-lg-3 "> </div> </div> </div> </div> <footer id="colophon" class="site-footer"> <div class="container"> <div class="row"> <div class="col-md-12 text-center"> <div class="site-info"> <span> Powered By: <a href="https://wordpress.org/" target="_blank">WordPress</a> </span> <span class="sep"> | </span> <span> Theme: <a href="https://odiethemes.com/themes/magazinebook/" target="_blank">MagazineBook</a> By OdieThemes </span> </div> </div> </div> </div> </footer> </div> <script>(function(){var advanced_ads_ga_UID="UA-88163215-1",advanced_ads_ga_anonymIP=!!1;window.advanced_ads_check_adblocker=function(t){var n=[],e=null;function a(t){var n=window.requestAnimationFrame||window.mozRequestAnimationFrame||window.webkitRequestAnimationFrame||function(t){return setTimeout(t,16)};n.call(window,t)}return a((function(){var t=document.createElement("div");t.innerHTML=" ",t.setAttribute("class","ad_unit ad-unit text-ad text_ad pub_300x250"),t.setAttribute("style","width: 1px !important; height: 1px !important; position: absolute !important; left: 0px !important; top: 0px !important; overflow: hidden !important;"),document.body.appendChild(t),a((function(){var a,o,i=null===(a=(o=window).getComputedStyle)||void 0===a?void 0:a.call(o,t),d=null==i?void 0:i.getPropertyValue("-moz-binding");e=i&&"none"===i.getPropertyValue("display")||"string"==typeof d&&-1!==d.indexOf("about:");for(var c=0,r=n.length;c<r;c++)n[c](e);n=[]}))})),function(t){"undefined"==typeof advanced_ads_adblocker_test&&(e=!0),null!==e?t(e):n.push(t)}}(),(()=>{function t(t){this.UID=t,this.analyticsObject="function"==typeof gtag;var n=this;return this.count=function(){gtag("event","AdBlock",{event_category:"Advanced Ads",event_label:"Yes",non_interaction:!0,send_to:n.UID})},function(){if(!n.analyticsObject){var e=document.createElement("script");e.src="https://www.googletagmanager.com/gtag/js?id="+t,e.async=!0,document.body.appendChild(e),window.dataLayer=window.dataLayer||[],window.gtag=function(){dataLayer.push(arguments)},n.analyticsObject=!0,gtag("js",new Date)}var a={send_page_view:!1,transport_type:"beacon"};window.advanced_ads_ga_anonymIP&&(a.anonymize_ip=!0),gtag("config",t,a)}(),this}advanced_ads_check_adblocker((function(n){n&&new t(advanced_ads_ga_UID).count()}))})();})();</script><div style="clear:both;width:100%;text-align:center; font-size:11px; "><a target="_blank" title="WP2Social Auto Publish" href="https://xyzscripts.com/wordpress-plugins/facebook-auto-publish/compare" >WP2Social Auto Publish</a> Powered By : <a target="_blank" title="PHP Scripts & Programs" href="http://www.xyzscripts.com" >XYZScripts.com</a></div><script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/navigation.js?ver=1.0.9" id="magazinebook-navigation-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/skip-link-focus-fix.js?ver=1.0.9" id="magazinebook-skip-link-focus-fix-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/jquery.easy-ticker.js?ver=3.1.0" id="magazinebook-news-ticker-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/splide.min.js?ver=2.3.1" id="splide-js-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/theme.js?ver=1.0.9" id="magazinebook-theme-js-js"></script> <script>!function(){window.advanced_ads_ready_queue=window.advanced_ads_ready_queue||[],advanced_ads_ready_queue.push=window.advanced_ads_ready;for(var d=0,a=advanced_ads_ready_queue.length;d<a;d++)advanced_ads_ready(advanced_ads_ready_queue[d])}();</script> </body> </html>