Blake
While you are able to run GPT Neo with just a CPU, do you want to? In this video, I explore how much time it takes to run the model on both the CPU and the GPU.
Notebook https://github.com/mallorbc/GPTNeo_notebook
GPT Neo GitHub https://github.com/EleutherAI/gpt-neo (use the first release tag)
GPT Neo HuggingFace docs https://huggingface.co/transformers/model_doc/gpt_neo.html
A useful article about transformer parameters https://huggingface.co/blog/how-to-generate
Timestamps
0:00 – Intro
1:25 – Explanation of the tests
2:04 – Loading model onto GPU test
3:03 – Loading the model onto the CPU test
4:12 – Model inferences on the CPU test
5:19 – Model inferences on the GPU test
6:31 – Comparing the results and discussion
8:09 – Outro