There is a newer version of this record available.

Software Open Access

GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow

Black, Sid; Leo, Gao; Wang, Phil; Leahy, Connor; Biderman, Stella

GPT-Neo is an implementation of model & data-parallel GPT-2 and GPT-3-like models, utilizing Mesh Tensorflow for distributed support. This codebase is designed for TPUs. It should also work on GPUs, though we do not recommend this hardware configuration.

Files (86.6 kB)
Name Size
EleutherAI/gpt-neo-v1.1.zip
md5:3b301a003caf2da94ae1c786c90d41d5
86.6 kB Download
1,182
34
views
downloads
All versions This version
Views 1,182880
Downloads 3423
Data volume 2.9 MB2.0 MB
Unique views 940770
Unique downloads 3423

Share

Cite as