ArXiV Archive
Description
This is a full archive of metadata about papers on arxiv.org from 1993-2018, including abstracts. Data is tidy and packed in TSV files, in two different collections of the total dataset: per year (all categories) and per primary category (all years). This archive also includes Jupyter notebooks for unpacking and analyzing it in python. See the README.md file and https://github.com/staeiou/arxiv_archive for more information.
Notes
In the Jupyter notebook 4-analysis-examples.ipynb, the per_year datasets are unpacked and merged to one large dataframe, which is then analyzed in various ways. If you are looking to use this data to do an analysis on the entire Arxiv, you may find this notebook useful to start.
These files are in processed_data/DUMP_DATE/per_year/YEAR.tsv.zip and processed_data/DUMP_DATE/per_category/CATEGORY_NAME.tsv.zip, with one row per line and tab-separated.
| Variable name | Definition | Example |
|---|---|---|
| abstract | Text of the abstract, may include LaTeX formatting. | We find the natural embedding of the (R+R^2)-i... |
| acm_class | ACM Classification (manually entered by authors, if exists) | |
| arxiv_id | Arxiv internal ID. Can get to PDF by appending "https://arxiv.org/pdf/" + arxiv_id + ".pdf" | 1011.0240 |
| author_text | Comma-separated list of authors | Sergei V. Ketov, Alexei A. Starobinsky |
| categories | Comma-separated list of all categories the paper was submitted to (posted and cross-posted) | hep-th,astro-ph.CO,gr-qc |
| comments | Author comments | 4 pages, revtex, no figures (very minor additi... |
| created | Date created (YYYY-MM-DD) | 2010-10-31 |
| doi | DOI (manually entered by authors, if exists) | 10.1103/PhysRevD.83.063512 |
| num_authors | Number of authors | 2 |
| num_categories | Number of categories | 3 |
| primary_cat | Primary category the paper was submitted to | hep-th |
| title | Paper title, may include LaTeX | Embedding (R+R^2)-Inflation into Supergravity |
| updated | Date last updated (YYYY-MM-DD) | 2011-02-28 |
| created_ym | Year and month created | 2010-10 |
Funding provided by: Alfred P. Sloan Foundation
Crossref Funder Registry ID: http://dx.doi.org/10.13039/100000879
Award Number: 2013-10-27
Funding provided by: Gordon and Betty Moore Foundation
Crossref Funder Registry ID: http://dx.doi.org/10.13039/100000936
Award Number: GBMF3834
Files
arxiv_archive_20190101.zip
Files
(1.5 GB)
| Name | Size | Download all |
|---|---|---|
|
md5:cf3025d42730d56e6c395c5549e7b9a6
|
1.5 GB | Preview Download |
Additional details
Related works
- Is derived from
- https://github.com/staeiou/arxiv_archive (URL)