#031 WE GOT ACCESS TO GPT-3! (With Gary Marcus, Walid Saba and Connor Leahy)

Machine Learning Street Talk

In this special edition, Dr. Tim Scarfe, Yannic Kilcher and Dr. Keith Duggar speak with Professor Gary Marcus, Dr. Walid Saba and Connor Leahy about GPT-3. We have all had a significant amount of time to experiment with GPT-3 and show you demos of it in use and the considerations. Do you think GPT-3 is a step towards AGI? Answer in the comments!

00:00:00 Connor’s take on LinkedIn
00:00:47 Show teaser
00:20:02 Tim Introduction
00:26:55 First look at GPT-3, python sorting
00:31:05 Search strategy in LMs
00:38:28 Character analogies and Melanie Mitchell
00:44:27 Substitution cipher
00:47:21 Database prompt
00:53:00 Broader Impact Generation
01:02:47 Gary Marcus Interview (Robust.AI)
01:29:11 Connor Leahy Interview (Eleuther.AI)
01:32:29 Connor — Tabular data
01:33:41 Connor — other surprising examples?
01:34:54 Connor — Is interpolated stuff new?
01:37:43 Connor — structure of the brain / How GPT works
01:41:21 Connor — Why cant GPT-3 reason?
01:46:30 Connor — Missing information problem and ideas on our our brains work
01:54:28 Connor — Topology of brain/models
01:58:49 Connor — Hardware lottery / LSTM / Transformer
02:01:41 Connor — NNs are just matrix program search
02:10:32 Connor — Google — information retrieval, the new paradigm, how to extract info from GPT-3, RL controller on top?
02:19:38 Connor — Database example / “pattern matching is Turing complete”
02:23:55 Connor — Did gpt3 understand?
02:26:30 Connor — Are the GOFAI people right?
02:27:40 Walid Saba on GPT-3
02:30:41 Walid — What is understanding and pattern recognition
02:35:56 Walid — Chomsky would be happy
02:42:13 Walid — Redefining success
02:46:05 Walid on Hinton
02:47:34 Walid on software 3.0
02:53:11 Keith — We use machine learning because we cant write code to do the same thing
02:59:36 Keith — What is pattern recognition and understanding
03:14:06 GPT-3 trials — Turing Dialog
03:15:35 GPT-3 trials — Mary Enjoyed a Sandwich
03:16:19 GPT-3 trials — BBC has five offices in Germany.
03:16:55 GPT-3 trials — Database prompt
03:20:23 GPT-3 trials — Python
03:20:31 GPT-3 trials — Patterns
03:21:01 GPT-3 trials — Database again
03:25:11 GPT-3 trials — GPT-3 experiment — the trophy doesn’t fit in the suitcase
03:27:32 GPT-3 trials — Scrambling words
03:30:41 GPT-3 trials — PDF cleanup example (Gwern)
03:35:03 GPT-3 trials — Word breaking and simple text patterns
03:37:16 GPT-3 trials — Typing of entities
03:38:30 GPT-3 trials — Basic Python append
03:39:07 GPT-3 trials — Automatic programming?
03:42:31 GPT-3 trials — Passive aggressive dialog input
03:44:39 GPT-3 trials — symptoms of depression
03:45:43 GPT-3 trials — Red shirts reasoning challenge
03:49:59 GPT-3 trials — Binary encoding
03:50:36 Concluding statements from Walid, Tim and Yannic

Pod version: https://anchor.fm/machinelearningstreettalk/episodes/031-WE-GOT-ACCESS-TO-GPT-3–With-Gary-Marcus–Walid-Saba-and-Connor-Leahy-en2h1k

Connor Leahy:
https://www.linkedin.com/in/connor-j-leahy/
https://twitter.com/NPCollapse
Eleuther.AI Discord — https://discord.com/invite/vtRgjbM

Gary Marcus:
https://www.linkedin.com/in/gary-marcus-b6384b4/
https://twitter.com/GaryMarcus
https://www.robust.ai

Walid Saba:
https://www.linkedin.com/in/walidsaba/
https://medium.com/ontologik
https://ontologik.ai

<iframe></p> <p><a href="https://www.youtube.com/watch?v=iccd86vOz3w">Source</a></p> <div class="be1e40beae42d993bafb8643f4ddde8b" data-index="3" style="float: none; margin:10px 0 10px 0; text-align:center;"> <script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script> <ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-layout="in-article" data-ad-format="fluid" data-ad-client="ca-pub-9244112244416304" data-ad-slot="4549240677"></ins> <script> (adsbygoogle = window.adsbygoogle || []).push({}); </script> </div> <div style="font-size: 0px; height: 0px; line-height: 0px; margin: 0; padding: 0; clear: both;"></div> </div> </article> <div class="clearfix"></div> <ul class="default-theme-post-navigation"> <li class="theme-nav-previous"><a href="https://theengineeringofconsciousexperience.com/polymer-developer-summit-2017-live-stream-day-1/" rel="prev"><span class="meta-nav">←</span> Polymer Developer Summit 2017 – Live Stream Day 1</a></li> <li class="theme-nav-next"><a href="https://theengineeringofconsciousexperience.com/conscious-ai-gpt-3-the-gnostic-jesus-christ-shorts/" rel="next">Conscious AI (GPT-3) & The Gnostic Jesus Christ #Shorts <span class="meta-nav">→</span></a></li> </ul> <div class="clearfix"></div> <h3 class='comment-reply-title'>Similar Posts</h3> <div class="mb-related-posts mb-simple-featured-posts mb-simple-featured-posts-wrap row"> <article class="mb-featured-article col-md-4 px-lg-3 post"> <a class="post-thumbnail" href="https://theengineeringofconsciousexperience.com/%d8%a7%d9%84%d8%b1%d8%a8%d8%ad-%d9%85%d9%86-%d8%a7%d9%84%d8%a7%d9%86%d8%aa%d8%b1%d9%86%d8%aa-%d9%88-%d8%a7%d9%84%d9%88%d8%b5%d9%88%d9%84-%d9%84%d9%84%d8%ad%d8%af-%d8%a7%d9%84%d8%a7%d8%af%d9%86%d9%89/" aria-hidden="true" tabindex="-1"> <img width="501" height="282" src="https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/11/1605909659_maxresdefault.jpg" class="attachment-magazinebook-featured-image-medium size-magazinebook-featured-image-medium wp-post-image" alt="" decoding="async" loading="lazy" srcset="https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/11/1605909659_maxresdefault.jpg 1280w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/11/1605909659_maxresdefault-300x169.jpg 300w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/11/1605909659_maxresdefault-1024x576.jpg 1024w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/11/1605909659_maxresdefault-768x432.jpg 768w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2020/11/1605909659_maxresdefault-520x293.jpg 520w" sizes="(max-width: 501px) 100vw, 501px" /> </a> <span class="cat-links"><a href="https://theengineeringofconsciousexperience.com/category/gpt-3/" rel="category tag">GPT 3</a></span> <header class="entry-header"> <h3 class="entry-title"><a href="https://theengineeringofconsciousexperience.com/%d8%a7%d9%84%d8%b1%d8%a8%d8%ad-%d9%85%d9%86-%d8%a7%d9%84%d8%a7%d9%86%d8%aa%d8%b1%d9%86%d8%aa-%d9%88-%d8%a7%d9%84%d9%88%d8%b5%d9%88%d9%84-%d9%84%d9%84%d8%ad%d8%af-%d8%a7%d9%84%d8%a7%d8%af%d9%86%d9%89/" rel="bookmark">الربح من الانترنت و الوصول للحد الادنى وسحب ارباحك قبل نهايه الفيديو</a></h3> <div class="entry-meta"> <span class="posted-on"><i class="far fa-calendar-alt"></i><a href="https://theengineeringofconsciousexperience.com/%d8%a7%d9%84%d8%b1%d8%a8%d8%ad-%d9%85%d9%86-%d8%a7%d9%84%d8%a7%d9%86%d8%aa%d8%b1%d9%86%d8%aa-%d9%88-%d8%a7%d9%84%d9%88%d8%b5%d9%88%d9%84-%d9%84%d9%84%d8%ad%d8%af-%d8%a7%d9%84%d8%a7%d8%af%d9%86%d9%89/" rel="bookmark"><time class="entry-date published updated" datetime="2020-11-18T13:00:06-07:00">November 18, 2020</time></a></span><span class="byline"><i class="far fa-user-circle"></i><span class="author vcard"><a class="url fn n" href="https://theengineeringofconsciousexperience.com/author/017d35079ace713ec70d08abea58a2a9/">Abdelrahman Ibrahem</a></span></span> </div> </header> </article> <article class="mb-featured-article col-md-4 px-lg-3 post"> <a class="post-thumbnail" href="https://theengineeringofconsciousexperience.com/gerika-sunday-school-23-mei-2021-gerika-gpt-pniel/" aria-hidden="true" tabindex="-1"> <img width="400" height="300" src="https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/06/1622561517_hqdefault.jpg" class="attachment-magazinebook-featured-image-medium size-magazinebook-featured-image-medium wp-post-image" alt="" decoding="async" loading="lazy" srcset="https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/06/1622561517_hqdefault.jpg 480w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/06/1622561517_hqdefault-300x225.jpg 300w" sizes="(max-width: 400px) 100vw, 400px" /> </a> <span class="cat-links"><a href="https://theengineeringofconsciousexperience.com/category/gpt-3/" rel="category tag">GPT 3</a></span> <header class="entry-header"> <h3 class="entry-title"><a href="https://theengineeringofconsciousexperience.com/gerika-sunday-school-23-mei-2021-gerika-gpt-pniel/" rel="bookmark">Gerika Sunday School – 23 Mei 2021 – GERIKA GPT PNIEL</a></h3> <div class="entry-meta"> <span class="posted-on"><i class="far fa-calendar-alt"></i><a href="https://theengineeringofconsciousexperience.com/gerika-sunday-school-23-mei-2021-gerika-gpt-pniel/" rel="bookmark"><time class="entry-date published updated" datetime="2021-05-22T18:00:14-07:00">May 22, 2021</time></a></span><span class="byline"><i class="far fa-user-circle"></i><span class="author vcard"><a class="url fn n" href="https://theengineeringofconsciousexperience.com/author/c907133f644f0ebb0ad953295ee41d15/">GPT PNIEL Bogor</a></span></span> </div> </header> </article> <article class="mb-featured-article col-md-4 px-lg-3 post"> <a class="post-thumbnail" href="https://theengineeringofconsciousexperience.com/que-es-mbr-y-gpt-y-en-cual-formato-se-utiliza/" aria-hidden="true" tabindex="-1"> <img width="400" height="300" src="https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/05/1620410576_hqdefault.jpg" class="attachment-magazinebook-featured-image-medium size-magazinebook-featured-image-medium wp-post-image" alt="" decoding="async" loading="lazy" srcset="https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/05/1620410576_hqdefault.jpg 480w, https://theengineeringofconsciousexperience.com/wp-content/uploads/2021/05/1620410576_hqdefault-300x225.jpg 300w" sizes="(max-width: 400px) 100vw, 400px" /> </a> <span class="cat-links"><a href="https://theengineeringofconsciousexperience.com/category/gpt-3/" rel="category tag">GPT 3</a></span> <header class="entry-header"> <h3 class="entry-title"><a href="https://theengineeringofconsciousexperience.com/que-es-mbr-y-gpt-y-en-cual-formato-se-utiliza/" rel="bookmark">Qué es MBR y GPT? y en cúal formato se utiliza?</a></h3> <div class="entry-meta"> <span class="posted-on"><i class="far fa-calendar-alt"></i><a href="https://theengineeringofconsciousexperience.com/que-es-mbr-y-gpt-y-en-cual-formato-se-utiliza/" rel="bookmark"><time class="entry-date published updated" datetime="2020-12-23T12:03:35-07:00">December 23, 2020</time></a></span><span class="byline"><i class="far fa-user-circle"></i><span class="author vcard"><a class="url fn n" href="https://theengineeringofconsciousexperience.com/author/248165666ed0b2b9dd73a9f283cdd384/">Chili</a></span></span> </div> </header> </article> </div> <div id="comments" class="comments-area"> <h5 class="comments-title"> 41 thoughts on “<span>#031 WE GOT ACCESS TO GPT-3! (With Gary Marcus, Walid Saba and Connor Leahy)</span>” </h5> <ol class="comment-list"> <li id="comment-200615" class="comment even thread-even depth-1"> <article id="div-comment-200615" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCmgy86ZVjrivlS7sJz844tg" class="url" rel="ugc external nofollow">aspie96</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200615"><time datetime="2020-11-27T18:56:35-07:00">November 27, 2020 at 6:56 pm</time></a> </div> </footer> <div class="comment-content"> <p><a href="https://www.youtube.com/watch?v=iccd86vOz3w&t=54m00s">54:00</a></p> <p>THANK YOU!!!!!</p> <p>Can we stop being absolute idiots about technology? We cannot NOW the broader impact. And there is no reason to expect the authors to know it better than others.</p> </div> </article> </li> <li id="comment-200614" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200614" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCMxnQ7gyb4BWKbLW5JWiaJA" class="url" rel="ugc external nofollow">Micheal Bee</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200614"><time datetime="2020-11-27T19:42:57-07:00">November 27, 2020 at 7:42 pm</time></a> </div> </footer> <div class="comment-content"> <p>FYI, count your prompt. It dropped one; so GPT-3 was doing what you asked.</p> </div> </article> </li> <li id="comment-200613" class="comment even thread-even depth-1"> <article id="div-comment-200613" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCTm-k6_SHqTrgZgipLnJa5w" class="url" rel="ugc external nofollow">Grigorii Guz</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200613"><time datetime="2020-11-27T22:21:26-07:00">November 27, 2020 at 10:21 pm</time></a> </div> </footer> <div class="comment-content"> <p>After watching both interviews with Walid, I still don't understand his point on probability in NLU. When someone says "I saw an elephant in my pijamas", either them or the elephant being in pajamas are both plausible meanings (but of course not equally probable, according to the listener's world model). So what's wrong with representing this probabilistically, especially when no additional context is available? And how can you even determine the exact thought of a person without hacking into their brain?</p> </div> </article> </li> <li id="comment-200612" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200612" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCllMxqTmPot1eCbSfGD7m1Q" class="url" rel="ugc external nofollow">Epos Nix</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200612"><time datetime="2020-11-28T01:06:14-07:00">November 28, 2020 at 1:06 am</time></a> </div> </footer> <div class="comment-content"> <p>It must sting to be these GOFAI-type guys that spent their entire lives trying to do what OpenAI managed to do in just a few years by just feeding tons of text into a transformer model. They're so quick to dismiss GPT-3 but have yet to show anything close to the holy grail they promise. Just look at Gary Marcus's website <a href="http://robust.ai/">robust.ai</a> "Building the world’s<br /> first industrial grade cognitive engine". Promising the moon but never delivering.</p> </div> </article> </li> <li id="comment-200611" class="comment even thread-even depth-1"> <article id="div-comment-200611" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCigwq47EQaL110A7kH8M9iw" class="url" rel="ugc external nofollow">Chris Gage</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200611"><time datetime="2020-11-28T01:09:01-07:00">November 28, 2020 at 1:09 am</time></a> </div> </footer> <div class="comment-content"> <p>Hey looking forward to GPT-42</p> </div> </article> </li> <li id="comment-200610" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200610" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC2q9pzZYK8wqzG1fwWeA84g" class="url" rel="ugc external nofollow">Abby</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200610"><time datetime="2020-11-28T01:28:06-07:00">November 28, 2020 at 1:28 am</time></a> </div> </footer> <div class="comment-content"> <p>Wow! Such an amazing video! The best video you have made 😍😍😍😍😍</p> </div> </article> </li> <li id="comment-200609" class="comment even thread-even depth-1"> <article id="div-comment-200609" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCxrR0xCKGHaGDAeaCZJA50g" class="url" rel="ugc external nofollow">Florian Hönicke</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200609"><time datetime="2020-11-28T01:56:11-07:00">November 28, 2020 at 1:56 am</time></a> </div> </footer> <div class="comment-content"> <p>Thanks for sharing. I'm always happy to see a new video coming up.</p> </div> </article> </li> <li id="comment-200608" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200608" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCmX5HBCUYth5rOS_MeSPNAw" class="url" rel="ugc external nofollow">Unknown TOM</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200608"><time datetime="2020-11-28T01:58:59-07:00">November 28, 2020 at 1:58 am</time></a> </div> </footer> <div class="comment-content"> <p><a href="https://www.youtube.com/watch?v=iccd86vOz3w&t=1h19m00s">1:19:00</a> but what if you born without eyes and legs and arms? What if all your input is someone describing you surrounding world using language?</p> </div> </article> </li> <li id="comment-200607" class="comment even thread-even depth-1"> <article id="div-comment-200607" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCmX5HBCUYth5rOS_MeSPNAw" class="url" rel="ugc external nofollow">Unknown TOM</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200607"><time datetime="2020-11-28T04:18:48-07:00">November 28, 2020 at 4:18 am</time></a> </div> </footer> <div class="comment-content"> <p>I think absent part is an inner dialog</p> </div> </article> </li> <li id="comment-200606" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200606" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCAbA0FivNSBqTmyYrh6uymQ" class="url" rel="ugc external nofollow">Eric Elliott</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200606"><time datetime="2020-11-28T05:13:47-07:00">November 28, 2020 at 5:13 am</time></a> </div> </footer> <div class="comment-content"> <p>Thanks for this video. Sorry if my reaction to Walid's episode was too harsh. I appreciate the skeptical arguments because they force me to think more robustly about the queries I am using, and the conclusions I draw from the responses.</p> <p>I have seen GPT-3 answer the corner table challenge correctly, BTW, conjuring people sitting at the table. An example using "coffee" and "table 3" is in a comment reply on the Walid episode.</p> <p>I have also seen it correctly produce output for generically-named functions, even with multiple layers of abstraction, using functions I wrote that don't show up in Google.</p> </div> </article> </li> <li id="comment-200605" class="comment even thread-even depth-1"> <article id="div-comment-200605" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC41kQqZ6bCbbBApX9BoaLew" class="url" rel="ugc external nofollow">quebono100</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200605"><time datetime="2020-11-28T05:38:46-07:00">November 28, 2020 at 5:38 am</time></a> </div> </footer> <div class="comment-content"> <p>4 hours love it 😛</p> </div> </article> </li> <li id="comment-200604" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200604" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCoUnpPCkHGuGFLtzNi4adcA" class="url" rel="ugc external nofollow">donrayjay</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200604"><time datetime="2020-11-28T05:55:51-07:00">November 28, 2020 at 5:55 am</time></a> </div> </footer> <div class="comment-content"> <p>The Case Against Reality by Donald Hoffman</p> </div> </article> </li> <li id="comment-200603" class="comment even thread-even depth-1"> <article id="div-comment-200603" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCoUnpPCkHGuGFLtzNi4adcA" class="url" rel="ugc external nofollow">donrayjay</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200603"><time datetime="2020-11-28T06:03:28-07:00">November 28, 2020 at 6:03 am</time></a> </div> </footer> <div class="comment-content"> <p>Thoughts don’t occur to me in words, they come in concepts and pictures. Is that unusual. An exception is when I am thinking specifically about a conversation that happened or that might happen in the future. Otherwise I don’t use words when thinking</p> </div> </article> </li> <li id="comment-200602" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200602" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC33c02d4_ssg0tGO3EXKG8g" class="url" rel="ugc external nofollow">Mark Ryan</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200602"><time datetime="2020-11-28T06:18:58-07:00">November 28, 2020 at 6:18 am</time></a> </div> </footer> <div class="comment-content"> <p>Epic episode – thanks so much for the balanced panel. It's also great that you were able to get access to GPT-3. Some experiments using GPT-3 for bread and butter tasks: <a href="https://youtu.be/790PiTSqi4Y">https://youtu.be/790PiTSqi4Y</a>, <a href="https://youtu.be/ECeRjLkT01U">https://youtu.be/ECeRjLkT01U</a>, <a href="https://youtu.be/MoLfVG-8Z5A">https://youtu.be/MoLfVG-8Z5A</a></p> </div> </article> </li> <li id="comment-200601" class="comment even thread-even depth-1"> <article id="div-comment-200601" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC41kQqZ6bCbbBApX9BoaLew" class="url" rel="ugc external nofollow">quebono100</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200601"><time datetime="2020-11-28T06:28:04-07:00">November 28, 2020 at 6:28 am</time></a> </div> </footer> <div class="comment-content"> <p>I knew it, GPT-3 is way overhypt!</p> </div> </article> </li> <li id="comment-200600" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200600" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC41kQqZ6bCbbBApX9BoaLew" class="url" rel="ugc external nofollow">quebono100</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200600"><time datetime="2020-11-28T06:33:00-07:00">November 28, 2020 at 6:33 am</time></a> </div> </footer> <div class="comment-content"> <p>Wow really nice tests</p> </div> </article> </li> <li id="comment-200599" class="comment even thread-even depth-1"> <article id="div-comment-200599" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC41kQqZ6bCbbBApX9BoaLew" class="url" rel="ugc external nofollow">quebono100</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200599"><time datetime="2020-11-28T06:56:25-07:00">November 28, 2020 at 6:56 am</time></a> </div> </footer> <div class="comment-content"> <p>Oh I have an nice thought about reasoning. One of my favorite author (Vera f. Birkenbihl) has a thought on inductive, deductive. She said, that we might explore a new reasoning which will be from a visual perspective. She also said, creativity is combining association which at the first look has no connections at all and then you have to reason about this new connection (comedians creating joke like this way)</p> </div> </article> </li> <li id="comment-200598" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200598" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC41kQqZ6bCbbBApX9BoaLew" class="url" rel="ugc external nofollow">quebono100</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200598"><time datetime="2020-11-28T07:22:31-07:00">November 28, 2020 at 7:22 am</time></a> </div> </footer> <div class="comment-content"> <p>Nice to include both camps of pro and contra GPT-3</p> </div> </article> </li> <li id="comment-200597" class="comment even thread-even depth-1"> <article id="div-comment-200597" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC41kQqZ6bCbbBApX9BoaLew" class="url" rel="ugc external nofollow">quebono100</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200597"><time datetime="2020-11-28T07:34:55-07:00">November 28, 2020 at 7:34 am</time></a> </div> </footer> <div class="comment-content"> <p>Connor Leahy doing a great job for humanity. I hate closed projects like GPT-3. Also DeepMind is very closed on theire development. And I admire George Hotz, he actually goes the other way. He put his ideas out on github and working with people who like to contributing and share it.</p> </div> </article> </li> <li id="comment-200596" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200596" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC41kQqZ6bCbbBApX9BoaLew" class="url" rel="ugc external nofollow">quebono100</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200596"><time datetime="2020-11-28T07:39:55-07:00">November 28, 2020 at 7:39 am</time></a> </div> </footer> <div class="comment-content"> <p>Sorry for spamming your comment section. But you got a good point GPT-3 has no feedback loop, where its reasoning about its own results. Like AlphaZero do.</p> </div> </article> </li> <li id="comment-200595" class="comment even thread-even depth-1"> <article id="div-comment-200595" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC41kQqZ6bCbbBApX9BoaLew" class="url" rel="ugc external nofollow">quebono100</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200595"><time datetime="2020-11-28T07:43:30-07:00">November 28, 2020 at 7:43 am</time></a> </div> </footer> <div class="comment-content"> <p><a href="https://www.youtube.com/watch?v=iccd86vOz3w&t=1h38m39s">1:38:39</a> Where did Yannic got this invisible making candies? xD</p> </div> </article> </li> <li id="comment-200594" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200594" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC41kQqZ6bCbbBApX9BoaLew" class="url" rel="ugc external nofollow">quebono100</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200594"><time datetime="2020-11-28T08:01:16-07:00">November 28, 2020 at 8:01 am</time></a> </div> </footer> <div class="comment-content"> <p>GPT-2 is repeatable, you can put an random seed in it.</p> </div> </article> </li> <li id="comment-200593" class="comment even thread-even depth-1"> <article id="div-comment-200593" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC5ESCua3AkV4wi8Vtso-FcQ" class="url" rel="ugc external nofollow">DavenH</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200593"><time datetime="2020-11-28T08:38:48-07:00">November 28, 2020 at 8:38 am</time></a> </div> </footer> <div class="comment-content"> <p>I really like Connor's style of discourse. His positions all seem to be buttressed by some first-principles argument.</p> </div> </article> </li> <li id="comment-200592" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200592" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC41kQqZ6bCbbBApX9BoaLew" class="url" rel="ugc external nofollow">quebono100</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200592"><time datetime="2020-11-28T08:52:31-07:00">November 28, 2020 at 8:52 am</time></a> </div> </footer> <div class="comment-content"> <p>You could try: text summarization, keyword extraction, put (Source:) at the end, try chess openings</p> </div> </article> </li> <li id="comment-200591" class="comment even thread-even depth-1"> <article id="div-comment-200591" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC5ESCua3AkV4wi8Vtso-FcQ" class="url" rel="ugc external nofollow">DavenH</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200591"><time datetime="2020-11-28T08:56:21-07:00">November 28, 2020 at 8:56 am</time></a> </div> </footer> <div class="comment-content"> <p>I'm worried that all three of the hosts are living in 4D and slipping in and out of our mortal 3D plane. Could be greenscreen artifacts, but I doubt it.</p> </div> </article> </li> <li id="comment-200590" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200590" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCuj62pvMVxDIgYuoya5usGw" class="url" rel="ugc external nofollow">ХОРОШО</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200590"><time datetime="2020-11-28T10:38:37-07:00">November 28, 2020 at 10:38 am</time></a> </div> </footer> <div class="comment-content"> <p>GPT-3 is a Chinese Room</p> </div> </article> </li> <li id="comment-200589" class="comment even thread-even depth-1"> <article id="div-comment-200589" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCDXY1x_hfEu2Fs0VlyN-Gtw" class="url" rel="ugc external nofollow">David Matthews</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200589"><time datetime="2020-11-28T11:36:24-07:00">November 28, 2020 at 11:36 am</time></a> </div> </footer> <div class="comment-content"> <p>have to say, I am more persuaded by Marcus' overall take than Connor's — GPT-3 is fun and impressive in many ways, but it really is a magic trick. And magic tricks are powerful and can perhaps give us insight into human weaknesses — clearly, this is a massively powerful pattern recognition tool that can generate interesting responses, because so much of what we do is grounded in simple patterns.</p> <p>But it is so incoherent and unconstrained as well. People are not generating just a bunch of "plausible" words in a row, and picking the optimal route. They have a personal story, an emotional state, personality, and a context they are embedded in (actually, we often have to negotiate several contexts at once) — and since it has none of that, it just spits out "convincing" text. </p> <p>It has no model of the world, no inner psychology, but equally importantly, all the intent and responsibility are still found in the human user.</p> <p>I think intent and responsibility are areas we'll need to think more about to get closer to AGI.</p> </div> </article> </li> <li id="comment-200588" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200588" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCiJdpJJ5d6osMxQ-ldE8X-Q" class="url" rel="ugc external nofollow">Carlos Segura</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200588"><time datetime="2020-11-28T20:07:48-07:00">November 28, 2020 at 8:07 pm</time></a> </div> </footer> <div class="comment-content"> <p>quality content 🔥🔥</p> </div> </article> </li> <li id="comment-200587" class="comment even thread-even depth-1"> <article id="div-comment-200587" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCMxnQ7gyb4BWKbLW5JWiaJA" class="url" rel="ugc external nofollow">Micheal Bee</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200587"><time datetime="2020-11-28T20:38:29-07:00">November 28, 2020 at 8:38 pm</time></a> </div> </footer> <div class="comment-content"> <p>I think what is being missed is that the brain does something like GPT, but it is done unconsciously. Historically, we've talked about connections to spirits, muses, and deities where ideas are formed in another dimension, but what if this is simply a background process using a wetware Hebbian model. The resulting inspiration is passed to the conscious mind where it can be evaluated and accepted or discarded. So imperceptibly, an idea prompted probabilistic formed stream is passed to the conscious mind. Reasoning is the loss function, and ego is the agent.</p> </div> </article> </li> <li id="comment-200586" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200586" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCDnWIiJWtrNVAvi_Jt1kRpg" class="url" rel="ugc external nofollow">SD Marlow</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200586"><time datetime="2020-11-28T22:07:06-07:00">November 28, 2020 at 10:07 pm</time></a> </div> </footer> <div class="comment-content"> <p>Already at odds with Conner and that is just from the teaser section.</p> </div> </article> </li> <li id="comment-200585" class="comment even thread-even depth-1"> <article id="div-comment-200585" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCQwJV1m9Tlq4Fy8NPVj01xw" class="url" rel="ugc external nofollow">wara wiri</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200585"><time datetime="2020-11-29T00:10:03-07:00">November 29, 2020 at 12:10 am</time></a> </div> </footer> <div class="comment-content"> <p>wow! really AMAZING video!! auto subs!!</p> </div> </article> </li> <li id="comment-200584" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200584" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC41kQqZ6bCbbBApX9BoaLew" class="url" rel="ugc external nofollow">quebono100</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200584"><time datetime="2020-11-29T10:05:16-07:00">November 29, 2020 at 10:05 am</time></a> </div> </footer> <div class="comment-content"> <p>I realized that chess openings are actually a good way to test GPT-3. On the one hand the internet is full of known chess openings, on the other hand, you can invent new chess openings. It would be interessting if GPT-3 can come up with new ones.</p> </div> </article> </li> <li id="comment-200583" class="comment even thread-even depth-1"> <article id="div-comment-200583" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCuriSFu8qOgovy7rTDIoJnA" class="url" rel="ugc external nofollow">Dan Jay</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200583"><time datetime="2020-11-29T13:08:07-07:00">November 29, 2020 at 1:08 pm</time></a> </div> </footer> <div class="comment-content"> <p>This must be The AI video of the year. It caused a massive brain shock 💥. Just like Tim said to Walid. I can never unlearn everything these guys unveiled. Thank you ❤</p> </div> </article> </li> <li id="comment-200582" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200582" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCORfBHY8voJt6DaRawKzD-w" class="url" rel="ugc external nofollow">Dan Frederiksen</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200582"><time datetime="2020-11-29T16:25:09-07:00">November 29, 2020 at 4:25 pm</time></a> </div> </footer> <div class="comment-content"> <p>I used to make the mistake of dismissing GPT2 as simply roughly verbatim recall from massive text with modest context matching and patching together. Some rudimentary adherence to human language form but a fancy parrot. I was wrong. He's quite wrong to think GPT3 is pure illusion. Yes it is limited, it is brittle, it's predicated on instant understanding, it can't mull things over but it simply could not do what it does without true understanding. That the understanding has limits and fails inelegantly does not negate it has understanding. I was surprised and shocked. I knew neural nets could basically do anything with clever application, I discovered that in late 2002 when I started deep learning. But I was working from the time efficient heuristic that none of you would do anything significant so I didn't have to pay attention 🙂 And it's true that GPT3 requires a vast curated dataset spun from decent intelligence but from that data it extracted a solid and real chunk of human level intelligence. It's not our architecture but it works and we weren't raised by wolves either. And its ability is further evidenced when it's single shot to other domains. Much of what it does I'm at a loss to explain. It's disturbing that it's in part catapulted to our level of intelligence and that taxes our abilities to comprehend. Now, this simple way to digest NL is not the final word in AI, it could well be somewhat of a dead end but it's just more blood in the water, it suddenly makes AI seem that much more attainable. I never published, Bush was president, yall are fools, careful what you wish for but it can be done. I happen to think NLP is the wrong place to start but kudos to openAI for taking it all the way and showing such a potent glimmer. Even if it is a dead end. It's a real dead end.<br />btw we obviously don't think in words. Language is much too poor to capture what's needed. NL is a weak representation needed for our low bandwidth comms. Also proven by animals and people without language. Even though the GPT architecture is nowhere nearly as sophisticated as ours, it demonstrates that with herculean effort you could extract real AI from us and the real world. And that might be what will happen. AI will slowly grow as inflexible but increasingly capable in all domains. Vastly super human in many aspects. I'm probably saying too much and wont be credited but it's entirely possible that a rigid slow learner could be trained so well that it could design a fast learning AI. Sigh, yall need Jesus. Fall to your knees and invite Jesus into your lives. And you all better be EV fans and vegetarians before year's end. I'm not kidding. This isn't the unattended kindergarten it appears to be, God and very potent ETs are playing along but do make an effort to be worthy of the power that is to come.</p> </div> </article> </li> <li id="comment-200581" class="comment even thread-even depth-1"> <article id="div-comment-200581" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC41kQqZ6bCbbBApX9BoaLew" class="url" rel="ugc external nofollow">quebono100</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200581"><time datetime="2020-11-29T16:33:55-07:00">November 29, 2020 at 4:33 pm</time></a> </div> </footer> <div class="comment-content"> <p>Please check out this video. <a href="https://www.youtube.com/watch?v=hRuBRHVbCB8">https://www.youtube.com/watch?v=hRuBRHVbCB8</a> It has interessting stuff in it</p> </div> </article> </li> <li id="comment-200580" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200580" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCLzZvnl-s5SMtsdL6LaRs7g" class="url" rel="ugc external nofollow">Frickin' CC deVille</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200580"><time datetime="2020-11-29T21:32:49-07:00">November 29, 2020 at 9:32 pm</time></a> </div> </footer> <div class="comment-content"> <p>Been waiting for G. Marcus' take on this. Thank you for posting</p> </div> </article> </li> <li id="comment-200579" class="comment even thread-even depth-1"> <article id="div-comment-200579" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC1nlF2LVjNHPUeB5d4s_Y2Q" class="url" rel="ugc external nofollow">Assaf Wodeslavsky</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200579"><time datetime="2020-11-30T05:05:18-07:00">November 30, 2020 at 5:05 am</time></a> </div> </footer> <div class="comment-content"> <p>“You should not believe that the magician is actually doing the trick”</p> </div> </article> </li> <li id="comment-200578" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200578" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCoOG1BblAFaydyjTeRM-Gjw" class="url" rel="ugc external nofollow">Paul Topping</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200578"><time datetime="2020-11-30T17:42:57-07:00">November 30, 2020 at 5:42 pm</time></a> </div> </footer> <div class="comment-content"> <p>Connor Leahy is just wrong. Innate knowledge is extremely important and, yes, evolution put it there over millions of years. He shrugs his shoulders and blows that off but I don't understand why. I don't know anything about Connor but he strikes me as a member of a certain category of scientist. He's got too much confidence. That's good in that it allows him to put huge amounts of energy into whatever he believes is important but only one in a billion will work on something that turns out to be valuable and correct. Could be Connor sees something no one else does but probably not.</p> </div> </article> </li> <li id="comment-200577" class="comment even thread-even depth-1"> <article id="div-comment-200577" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCQ7b19zM6Opwps4DT6YEVDw" class="url" rel="ugc external nofollow">Eric Mican</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200577"><time datetime="2020-12-01T05:01:51-07:00">December 1, 2020 at 5:01 am</time></a> </div> </footer> <div class="comment-content"> <p>Was listening to Marcus and thinking if nothing else, GPT-3 is a milestone in training infrastructure</p> </div> </article> </li> <li id="comment-200576" class="comment odd alt thread-odd thread-alt depth-1"> <article id="div-comment-200576" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UC8YI1_APwk42erokrUd_ygQ" class="url" rel="ugc external nofollow">Cristian Garcia</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200576"><time datetime="2020-12-01T08:48:59-07:00">December 1, 2020 at 8:48 am</time></a> </div> </footer> <div class="comment-content"> <p>1: I think its really easy to point out the limitations of current approaches and state that they are not the holy grail while at the same time not giving a (good) alternative. Saying NLP != NLU has 0 impact, at least people like Bengio or Lecun point the limitations while still giving a realistic agenda (e.g. energy based models).<br />2. I have the following opinion: GPT3 is the most general AI we have right now, it may be very weak, but we have nothing like it. A single algorithm can do sentiment analysis, information retrieval, pattern matching, ect. I think animals (including humans) are much like this, very bad at doing highly specific tasks but very good at giving a good guess at something unknown. I think this is much more worthwhile and can be tuned to many commercial applications than trying to specify what the "thought of a sentence" means.</p> </div> </article> </li> <li id="comment-200575" class="comment even thread-even depth-1"> <article id="div-comment-200575" class="comment-body"> <footer class="comment-meta"> <div class="comment-author vcard"> <b class="fn"><a href="https://www.youtube.com/channel/UCxDP_JlV1bw0lahXbSt9zKg" class="url" rel="ugc external nofollow">Keith Duggar</a></b> <span class="says">says:</span> </div> <div class="comment-metadata"> <a href="https://theengineeringofconsciousexperience.com/031-we-got-access-to-gpt-3-with-gary-marcus-walid-saba-and-connor-leahy/#comment-200575"><time datetime="2020-12-01T11:44:45-07:00">December 1, 2020 at 11:44 am</time></a> </div> </footer> <div class="comment-content"> <p>Dear Internet,</p> <p>DNNs of any flavor, including RNNs, are not Turing Machines (TM) and are not Turing complete. For those who want nice pictures and a thorough explanation, this Stack Overflow post is correct:</p> <p><a href="https://stackoverflow.com/a/53022636">https://stackoverflow.com/a/53022636</a></p> <p>One practical manifestation of this fact is that DNNs are obscenely inefficient, requiring vast (or infinite) numbers of circuits (nodes, weights, precision, etc) to compute functions that could be computed by finite programs running on a TMs/computers. For example, think of a DNN that could output the Nth digit of Pi. Given what we know about Pi today, such a DNN would require an actual infinity of circuits whereas one can write a finite program that will terminate in a finite number of steps for any N on a Turing machine.</p> </div> </article> </li> </ol> <p class="no-comments">Comments are closed.</p> </div> </main> </div> <div class="col-md-3 px-lg-3 "> </div> </div> </div> </div> <footer id="colophon" class="site-footer"> <div class="container"> <div class="row"> <div class="col-md-12 text-center"> <div class="site-info"> <span> Powered By: <a href="https://wordpress.org/" target="_blank">WordPress</a> </span> <span class="sep"> | </span> <span> Theme: <a href="https://odiethemes.com/themes/magazinebook/" target="_blank">MagazineBook</a> By OdieThemes </span> </div> </div> </div> </div> </footer> </div> <script>(function(){var advanced_ads_ga_UID="UA-88163215-1",advanced_ads_ga_anonymIP=!!1;window.advanced_ads_check_adblocker=function(t){var n=[],e=null;function a(t){var n=window.requestAnimationFrame||window.mozRequestAnimationFrame||window.webkitRequestAnimationFrame||function(t){return setTimeout(t,16)};n.call(window,t)}return a((function(){var t=document.createElement("div");t.innerHTML=" ",t.setAttribute("class","ad_unit ad-unit text-ad text_ad pub_300x250"),t.setAttribute("style","width: 1px !important; height: 1px !important; position: absolute !important; left: 0px !important; top: 0px !important; overflow: hidden !important;"),document.body.appendChild(t),a((function(){var a,o,i=null===(a=(o=window).getComputedStyle)||void 0===a?void 0:a.call(o,t),d=null==i?void 0:i.getPropertyValue("-moz-binding");e=i&&"none"===i.getPropertyValue("display")||"string"==typeof d&&-1!==d.indexOf("about:");for(var c=0,r=n.length;c<r;c++)n[c](e);n=[]}))})),function(t){"undefined"==typeof advanced_ads_adblocker_test&&(e=!0),null!==e?t(e):n.push(t)}}(),(()=>{function t(t){this.UID=t,this.analyticsObject="function"==typeof gtag;var n=this;return this.count=function(){gtag("event","AdBlock",{event_category:"Advanced Ads",event_label:"Yes",non_interaction:!0,send_to:n.UID})},function(){if(!n.analyticsObject){var e=document.createElement("script");e.src="https://www.googletagmanager.com/gtag/js?id="+t,e.async=!0,document.body.appendChild(e),window.dataLayer=window.dataLayer||[],window.gtag=function(){dataLayer.push(arguments)},n.analyticsObject=!0,gtag("js",new Date)}var a={send_page_view:!1,transport_type:"beacon"};window.advanced_ads_ga_anonymIP&&(a.anonymize_ip=!0),gtag("config",t,a)}(),this}advanced_ads_check_adblocker((function(n){n&&new t(advanced_ads_ga_UID).count()}))})();})();</script><div style="clear:both;width:100%;text-align:center; font-size:11px; "><a target="_blank" title="WP2Social Auto Publish" href="https://xyzscripts.com/wordpress-plugins/facebook-auto-publish/compare" >WP2Social Auto Publish</a> Powered By : <a target="_blank" title="PHP Scripts & Programs" href="http://www.xyzscripts.com" >XYZScripts.com</a></div><script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/navigation.js?ver=1.0.9" id="magazinebook-navigation-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/skip-link-focus-fix.js?ver=1.0.9" id="magazinebook-skip-link-focus-fix-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/jquery.easy-ticker.js?ver=3.1.0" id="magazinebook-news-ticker-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/splide.min.js?ver=2.3.1" id="splide-js-js"></script> <script type="text/javascript" src="https://theengineeringofconsciousexperience.com/wp-content/themes/magazinebook/js/theme.js?ver=1.0.9" id="magazinebook-theme-js-js"></script> <script>!function(){window.advanced_ads_ready_queue=window.advanced_ads_ready_queue||[],advanced_ads_ready_queue.push=window.advanced_ads_ready;for(var d=0,a=advanced_ads_ready_queue.length;d<a;d++)advanced_ads_ready(advanced_ads_ready_queue[d])}();</script> </body> </html>