There is a newer version of this record available.

Software Open Access

GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow

Black, Sid; Leo, Gao; Wang, Phil; Leahy, Connor; Biderman, Stella

GPT-Neo is an implementation of model & data-parallel GPT-2 and GPT-3-like models, utilizing Mesh Tensorflow for distributed support. This codebase is designed for TPUs. It should also work on GPUs, though we do not recommend this hardware configuration.

Files (86.6 kB)
Name Size
EleutherAI/gpt-neo-v1.1.zip
md5:3b301a003caf2da94ae1c786c90d41d5
86.6 kB Download
2,521
71
views
downloads
All versions This version
Views 2,5211,654
Downloads 7151
Data volume 6.2 MB4.4 MB
Unique views 2,0491,466
Unique downloads 7050

Share

Cite as