Presentation Open Access

PIDs, Petabytes and Neutrons

Gareth Murphy

Neutron science generates an exploding volume of scientific research data which needs to be managed under FAIR data principles. The European Spallation Source (ESS), is expected to generate tens of petabytes of data per year.

This volume of scientific research data benefits from a “PIDcentric” approach to manage the many science users, data sets and instruments.

In partnership with the Swiss and Swedish national synchrotron radiation facilities at PSI and MAXIV, ESS have developed SciCat, a new data catalogue. SciCat allows users to register and access data using persistent identifiers.

SciCat users are identified using the ORCID database, published datasets are identified using DOIs and  for raw and derived datasets, identifiers are used.

SciCat uses a document-oriented database, MongoDB, which allows datasets to be tagged with ad hoc, unstructured scientific metadata as well as traditionally structured metadata, which can then be recovered using the DOI or Handle.

In the day to day operations, SciCat also has to deal with the problem of legacy scientific research data. Legacy data with incomplete or lost metadata presents a challenge to include in data catalogue. In my session, I will talk about the ESS’s experiences with PIDs, data catalogues and legacy data.

Files (20.4 MB)
Name Size
20.4 MB Download
All versions This version
Views 341340
Downloads 126126
Data volume 2.6 GB2.6 GB
Unique views 322321
Unique downloads 112112


Cite as