NetCDF at the 4TU.Centre for Research Data

The 4TU.Centre for Research Data (short version: 4TU.ResearchData) was started in 2008 as a collaboration of the libraries of three universities of technology in the Netherlands: Delft University of Technology, Eindhoven University of Technology, and the University of Twente. The data archive, which has been fully operational since about 2010, collects data in science and engineering in a permanent and sustainable manner.

Presently, around 90% of the data (both in terms of volume and number of datasets) stored in the archive are atmospheric and environmental research datasets coded in netCDF – a data format and model that, although generic, is mainly and widely used in climate and atmospheric sciences and oceanography.

4TU.ResearchData has a special interest in this area and it offers specific services and tools to enhance the access to netCDF datasets. In particular, netCDF files can be accessed via the OPeNDAP (Open-source Project for a Network Data Access Protocol) protocol, the main advantage of which is the ability to retrieve subsets of files without the need to download whole datasets.

To better understand netCDF data at 4TU.ResearchData – the datasets and their contributors as well as the services and their users – we conducted desk-based research and a series of semi-structured qualitative interviews with researchers based in the Netherlands who use and produce netCDF data.

This report provides an overview of the current data and services and explores options for 4TU.ResearchData to expand its services related to netCDF data. The analysis is broad in scope, assessing  opportunities for creating not just technical services related to storing and archiving netCDF data, but also for advice and guidance, and the advantages that could accrue from building a community of data depositors and users.

Our main conclusions are that the creators and users of the netCDF data stored in 4TU.ResearchData represent heterogeneous research communities within the Earth sciences. They have different views and attitudes to data archiving and data publishing, and store netCDF datasets with very different spatio-temporal characteristics in the archive. Ensuring that any new and current netCDF services continue to be relevant to these communities will require taking this diversity into account.

A need for training and guidance – particularly on data management aspects related to documentation, metadata standards and conventions – is the common thread uniting these communities. This will provide the way forward for 4TU.ResearchData to build a community of data depositors and users.

Expanding technical services beyond what is already provided might be more difficult, given the diversity of the data and the depositors and users, but there are a few services that could help support community building efforts, with the desired goal being higher-quality data and increased rates of data reuse.