Published February 26, 2022
| Version v1
Dataset
Open
A Systematic Evaluation of Large Language Models of Code
Authors/Creators
- 1. Carnegie Mellon University
Description
Test sets of ~100 files in each of 12 programming languages.
These files are not included in The Pile, and thus models such as GPT-Neo, GPT-J, GPT-NeoX were not trained on them.
In the paper, we use these test sets to compare a variety of language models of code including OpenAI's Codex, GPT-J, GPT-Neo, GPT-NeoX-20B, and CodeParrot and our PolyCoder model.
Notes
https://arxiv.org/abs/2202.13169
Files
Files
(904.1 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:8d8b890d2cfa7d26530b028f1a6fa47e
|
904.1 kB | Download |