Henry AI Labs
This video will explore the exciting new 6.8 Billion parameter ImageGPT model! The researchers show that better and larger generative models learn better representations for tasks like ImageNet classification!
Thanks for watching! Please Subscribe!
Paper Links:
ImageGPT (Blog Post): https://openai.com/blog/image-gpt/
ImageGPT (Paper): https://cdn.openai.com/papers/Generative_Pretraining_from_Pixels_V2.pdf
A Survey of Long-term Context in Transformers: https://www.pragmatic.ml/a-survey-of-methods-for-incorporating-long-term-context/
Google TPUs: https://cloud.google.com/tpu/docs/tpus
The Illustrated Transformer: http://jalammar.github.io/illustrated-transformer/
PixelCNN: https://keras.io/examples/generative/pixelcnn/
PixelCNN (Paper): https://arxiv.org/pdf/1606.05328.pdf
Contrastive Predictive Coding: https://arxiv.org/pdf/1905.09272.pdf
Big BiGAN: https://arxiv.org/pdf/1907.02544.pdf
BERT: https://arxiv.org/pdf/1810.04805.pdf
Rethinking Pre-training and Self-Training: https://arxiv.org/pdf/2006.06882.pdf