A Science Gateway for climate data analysis based on Virtual Analysis Ready Data

The Earth System Grid Federation (ESGF) Virtual Aggregation aims to enhance the accessibility and facilitate climate data analysis of datasets housed within the ESGF, a globally distributed network of research centres collaborating as a federated data archive. The ESGF provides modelling groups with nodes for publishing and archiving their model outputs to make them accessible to the climate community at any time. The federation has been chosen as a platform for dissemination of climate data of international initiatives such as CMIP and CORDEX. Thus, the ESGF has become the reference archive for Assessment Reports (AR) on Climate Change produced by the Intergovernmental Panel on Climate Change (IPCC). However, use cases in scientific research often involve calculations on datasets spanning multiple variables, over the whole time period and multiple model ensembles.

The ESGF Virtual Aggregation extends the federation capabilities, beyond file search and download, by providing out of the box remote climate data analysis capabilities over data analysis ready, virtually aggregated, climate datasets, on top of the existing software stack of the federation. In contrast to other new methodologies based on the creation of Analysis Ready Data (ARD) and physical duplication of the datasets of the ESGF, this work presents a methodology based on logical and virtual aggregations that does not require duplication of the data. Moreover, this work exploits all the existing data services already available in the federation. Thus, climate analysis ready data is provided in a much more sustainable way.

The ESGF Virtual Aggregation is provided both as a dataset and as a science gateway for users interested in climate data analysis. This work describes the methodology, implementation and features of the ESGF Virtual Aggregation. Also, a comparison between additional methods for the generation of climate analysis ready data within the context of the ESGF is provided. The trade-offs of the different approaches are considered and a comparison of climate data analysis use cases will be presented.