Machine Learning Street Talk
In this special edition, Dr. Tim Scarfe, Yannic Kilcher and Dr. Keith Duggar speak with Professor Gary Marcus, Dr. Walid Saba and Connor Leahy about GPT-3. We have all had a significant amount of time to experiment with GPT-3 and show you demos of it in use and the considerations. Do you think GPT-3 is a step towards AGI? Answer in the comments!
00:00:00 Connor’s take on LinkedIn
00:00:47 Show teaser
00:20:02 Tim Introduction
00:26:55 First look at GPT-3, python sorting
00:31:05 Search strategy in LMs
00:38:28 Character analogies and Melanie Mitchell
00:44:27 Substitution cipher
00:47:21 Database prompt
00:53:00 Broader Impact Generation
01:02:47 Gary Marcus Interview (Robust.AI)
01:29:11 Connor Leahy Interview (Eleuther.AI)
01:32:29 Connor — Tabular data
01:33:41 Connor — other surprising examples?
01:34:54 Connor — Is interpolated stuff new?
01:37:43 Connor — structure of the brain / How GPT works
01:41:21 Connor — Why cant GPT-3 reason?
01:46:30 Connor — Missing information problem and ideas on our our brains work
01:54:28 Connor — Topology of brain/models
01:58:49 Connor — Hardware lottery / LSTM / Transformer
02:01:41 Connor — NNs are just matrix program search
02:10:32 Connor — Google — information retrieval, the new paradigm, how to extract info from GPT-3, RL controller on top?
02:19:38 Connor — Database example / “pattern matching is Turing complete”
02:23:55 Connor — Did gpt3 understand?
02:26:30 Connor — Are the GOFAI people right?
02:27:40 Walid Saba on GPT-3
02:30:41 Walid — What is understanding and pattern recognition
02:35:56 Walid — Chomsky would be happy
02:42:13 Walid — Redefining success
02:46:05 Walid on Hinton
02:47:34 Walid on software 3.0
02:53:11 Keith — We use machine learning because we cant write code to do the same thing
02:59:36 Keith — What is pattern recognition and understanding
03:14:06 GPT-3 trials — Turing Dialog
03:15:35 GPT-3 trials — Mary Enjoyed a Sandwich
03:16:19 GPT-3 trials — BBC has five offices in Germany.
03:16:55 GPT-3 trials — Database prompt
03:20:23 GPT-3 trials — Python
03:20:31 GPT-3 trials — Patterns
03:21:01 GPT-3 trials — Database again
03:25:11 GPT-3 trials — GPT-3 experiment — the trophy doesn’t fit in the suitcase
03:27:32 GPT-3 trials — Scrambling words
03:30:41 GPT-3 trials — PDF cleanup example (Gwern)
03:35:03 GPT-3 trials — Word breaking and simple text patterns
03:37:16 GPT-3 trials — Typing of entities
03:38:30 GPT-3 trials — Basic Python append
03:39:07 GPT-3 trials — Automatic programming?
03:42:31 GPT-3 trials — Passive aggressive dialog input
03:44:39 GPT-3 trials — symptoms of depression
03:45:43 GPT-3 trials — Red shirts reasoning challenge
03:49:59 GPT-3 trials — Binary encoding
03:50:36 Concluding statements from Walid, Tim and Yannic
Connor Leahy:
https://www.linkedin.com/in/connor-j-leahy/
https://twitter.com/NPCollapse
Eleuther.AI Discord — https://discord.com/invite/vtRgjbM
Gary Marcus:
https://www.linkedin.com/in/gary-marcus-b6384b4/
https://twitter.com/GaryMarcus
https://www.robust.ai
Walid Saba:
https://www.linkedin.com/in/walidsaba/
https://medium.com/ontologik
https://ontologik.ai