D3A: Direct DOI Dataset Access – Streamlining Data Discovery

Within the OSCARS project, we recognize that associating datasets with persistent identifiers (a globally unique ID) is crucial for ensuring that research outputs are available for open science and comply with the FAIR principles (Findable, Accessible, Interoperable, and Reusable). This enhances the value of the data and it also allows other people to recreate existing work, or to use the data in new and novel research.

Currently, several platforms and institutes offer the possibility to store data with a DOI, a type of persistent identifier. However, no standard exists that allows a machine to access or download the data from a DOI. Repositories typically either have no way to download data automatically, or have adopted some proprietary solution. With no widely deployed standard, any support for accessing or downloading data from a DOI can only be incomplete.

We have successfully developed a proof-of-concept code based on existing approaches in other contexts (HTTP content-negotiation and metalink) that demonstrates the applicability of our approach. This involves a mixture of modifying existing production software, like the fsspec python module, and developing new code to build a demonstration of the benefits of this approach.

Given our code's foundation in the fsspec Python module, its potential applications are vast and evident. It can be seamlessly integrated into various Python applications, such as Jupyter Notebooks, and is particularly well-suited for use cases where the pandas module is already employed, as pandas imports fsspec by default. Furthermore, our code has significant potential for integration within the European Open Science Cloud (EOSC), where it can facilitate dataset access and reuse across disciplines and borders, thereby enhancing the EOSC's mission to provide a unified and seamless environment for data-driven research.