GPT 3

Can a language model (like GPT-3) learn meaning? But really?



AI Coffee Break with Letitia Parcalabescu

What is meaning? How far is state-of-the art NLP from meaning? Do publications misuse the word “meaning”?

📺 Transformer combining Vision and Language: https://youtu.be/dd7nE4nbxN0
📺 GPT-3 video: https://youtu.be/5fqxPOaaqi0
📺 BERTology meets Biology: https://youtu.be/pFf4PltQ9LY

Outline:
* 00:00 500 subs celebration!
* 00:53 Breathing pause
* 02:12 Misuse of “meaning”
* 04:22 What is meaning?
* 06:39 Language models and meaning
* 08:20 Octopus experiment
* 11:09 What to do?

📄 Paper: Bender, Emily M., and Alexander Koller. “Climbing towards NLU: On meaning, form, and understanding in the age of data.” In Proc. of ACL. 2020. https://www.aclweb.org/anthology/2020.acl-main.463/

🎧 Audiopaper: https://soundcloud.com/emily-m-bender/climbingtowardsnlu-audiopaper/s-0ZT7112K1Ep

🔗 Links:
YouTube: https://www.youtube.com/aicoffeebreakwithletitiaparcalabescu
Twitter: https://twitter.com/AICoffeeBreak
Reddit: https://www.reddit.com/r/AICoffeeBreak/

#AICoffeeBreak #MsCoffeeBean #ACL2020