Report on the final implementation of key selected post-processing, analytics and visualisation applications and main outcomes - Deliverable D5.3

In the ESiWACE Center of Excellence the Earth System Data Middleware (ESDM) provides a data model that manages datasets in virtual containers with their metadata.

Building on this work, a Python client has been developed for building pipelines consisting of ESDM-enabled post-processing, analytics and visualisation (PAV) applications. Among these PAV applications the key selected ones are: Ophidia, an open source solution for the analysis of multi-dimensional data, mainly targeted at climate science applications; CDO, a collection of command line operators to manipulate and analyse climate and weather data; ParaView, a widely used application for interactive 3D rendering and analysis of large data sets. The Ophidia framework has been improved in various ways, in particular with respect to parallel I/O, handling of non-regular grids and in-flight analytics. For the CDOs we have shown that a significant performance improvement can be obtained using GPUs. ParaView now supports arbitrary grids, such as those of the ICON, IFS, and FESOM models, the components of the DestinE digital twins. With the newly developed YAVIZ interface, it can also be used for online (in-transit) diagnostics and visualization with models that use the YAC coupler (currently the ICON model, we are working on YAC-support in FESOM). Furthermore, we have developed Python bindings for YAC, so now any Python based analysis (e.g. machine learning codes) can be applied as online analysis to the fields of the ICON model.