Dataset Open Access

Zenodo metadata JSON records as of 2019-09-16

Stian Soiland-Reyes; Paul Groth

This preliminary dataset contains the application/vnd.zenodo.v1+json JSON records of Zenodo deposits as retrieved on 2019-09-16.


  • zenodo-records-json-2019-09-16.tar.xz Zenodo JSON records
    XZ-compressed tar archive of individual JSON records as retrieved from Zenodo. Filenames reflects record, e.g. 1310621.json was retrieved from using content-negotiation for application/vnd.zenodo.v1+json
  • zenodo-records-json-2019-09-16-filtered.jsonseq.xz Concatinated Zenodo JSON records
    XZ-compressed RFC7464 JSON Sequence stream, readable by jq. Concatination of Zenodo JSON records. Order not significant.
  • Retrieve Zenodo JSON records
    A retrospectively created Bash shell script that shows the commands used to retrieve JSON files and concationate to jsonseq.
  • ro-crate-metadata.jsonld RO-Crate 0.2 structured metadata
  • ro-crate-preview.html Browser rendering of RO-Crate structured metadata
  • This dataset description


This dataset is provided under the license Apache License, version 2.0:

Copyright 2019 The University of Manchester

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

CC0 for Zenodo metadata

The Zenodo metadata in zenodo-records-json-2019-09-16.tar.xz is reused under the terms of


To retrieve the Zenodo JSON it was deemed necessary to use the undocumented parts of Zenodo API.

From the Zenodo source code it was identified that the REST template{pid_value} could be used with pid_value as the numeric part from the OAI-PMH identifier, e.g. for the Zenodo JSON can be retrieved at

The JSON API supports content negotiation, the content-types supported as of 2019-09-20 include:

Using these (currently) undocumented parts of the Zenodo API thus avoids the need for HTML scraping while also giving individual complete records that are suitable to redistribute as records in a filtered dataset.

This preliminary exploration will be adapted into the reproducible CWL workflow, for now included as a Bash script

Execution time was about 3 days from a server at the University of Manchester network on a single 1 GBps network link. The script does:

  • Retrieve each of the first 3.5 million Zenodo records
    as Zenodo JSON by iterating over possible numeric IDs (the maximum ID 3450000 was estimated from "Recent uploads")
  • Filter list to exclude records that are not found, moved or deleted. The presence of the key conceptrecid is used as marker.
  • Use jq to ensure the JSON is on a single line
  • Join the JSON files using the ASCII Record Separator (RS, 0x1e) to make a application/json-seq JSON text sequence stream
  • Save the JSON stream as a single compressed file using xz
Files (1.2 GB)
Name Size
4.7 kB Download
8.0 kB Download
22.6 kB Download
521.4 MB Download
664.4 MB Download
1.8 kB Download
All versions This version
Views 235234
Downloads 255255
Data volume 108.0 GB108.0 GB
Unique views 213212
Unique downloads 8080


Cite as