AI Coffee Break with Letitia
Ms. Coffee Bean explains how a huge collaboration of researchers managed to extract training data from large language models like GPT-2. Why is this even possible and what does this mean for even larger models like GPT-3?
” Extracting Training Data from Large Language Models” explained by Carlini et al. 2020.
📺 GPT-3 – what you need to know: https://youtu.be/5fqxPOaaqi0
Outline:
* 00:00 Large Language Models
* 01:55 GPT-2
* 02:33 Why is it possible?
* 03:38 k-eidetic memorization
* 04:21 How does the attack work?
* 05:49 How bad is it?
* 08:12 What to do?
📄 Paper explained: Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., … & Oprea, A. (2020). Extracting Training Data from Large Language Models. arXiv preprint arXiv:2012.07805. https://arxiv.org/pdf/2012.07805.pdf
————————————————————-
🔗 Links:
YouTube: https://www.youtube.com/AICoffeeBreak
Twitter: https://twitter.com/AICoffeeBreak
Reddit: https://www.reddit.com/r/AICoffeeBreak/
#AICoffeeBreak #MsCoffeeBean #GPT #MachineLearning #AI #research