Presentation Open Access

PIDs, Petabytes and Neutrons

Gareth Murphy

Neutron science generates an exploding volume of scientific research data which needs to be managed under FAIR data principles. The European Spallation Source (ESS), is expected to generate tens of petabytes of data per year.

This volume of scientific research data benefits from a “PIDcentric” approach to manage the many science users, data sets and instruments.

In partnership with the Swiss and Swedish national synchrotron radiation facilities at PSI and MAXIV, ESS have developed SciCat, a new data catalogue. SciCat allows users to register and access data using persistent identifiers.

SciCat users are identified using the ORCID database, published datasets are identified using DOIs and  for raw and derived datasets, handle.net identifiers are used.

SciCat uses a document-oriented database, MongoDB, which allows datasets to be tagged with ad hoc, unstructured scientific metadata as well as traditionally structured metadata, which can then be recovered using the DOI or Handle.

In the day to day operations, SciCat also has to deal with the problem of legacy scientific research data. Legacy data with incomplete or lost metadata presents a challenge to include in data catalogue. In my session, I will talk about the ESS’s experiences with PIDs, data catalogues and legacy data.

Files (20.4 MB)
Name Size
20190124pidapalooza.pdf
md5:3f852639785a893d357ebe3f714843d0
20.4 MB Download
104
72
views
downloads
All versions This version
Views 104104
Downloads 7272
Data volume 1.5 GB1.5 GB
Unique views 9898
Unique downloads 6464

Share

Cite as