Enhancing Earth system model evaluation with data cube enabled machine learning

Machine learning (ML) techniques represent a promising avenue to enhance climate model evaluation, better understand Earth system processes and further improve climate modeling. The application of ML techniques on multivariate climate data with high temporal and spatial frequencies may lead to several technical challenges, a major issue often being that input data can significantly exceed the memory available on compute systems. This data challenge can be circumvented by relying on cloud ready data which allows processing of data in a memory efficient way and unlocks the application of ML methods on large input datasets. In this pilot project, the Earth System Model Evaluation Tool (ESMValTool) was extended by interfacing it with cloud-based analysis-ready data streams from the Earth system data cube infrastructure. In a second step, a ML-based analysis package was coupled to ESMValTool to demonstrate the integration of ML algorithms for climate model evaluation, with a particular focus on causal discovery applied to Arctic-midlatitude teleconnections. The main beneficiaries of this pilot are the Earth system science community, including climate model development and evaluation groups, the climate informatics community and infrastructure providers such as High Performance Computing (HPC) centers and science data providers. This pilot opens up a promising avenue towards the efficient handling of Earth system data for application of ML methods which will benefit the NFDI4Earth community.