VisMetHack2022: Visualizing winds and surface variables from the ECMWF IFS 1-km nature run

Overview

This data collection was contributed to the Visualisation Hackathon 2022 (#VisMetHack2022), in conjunction with the Using ECMWF's Forecasts (UEF2022) workshop.

The European Center for Medium-Range Weather Forecasts (ECMWF) and the Oak Ridge National Laboratory (ORNL) are pleased to announce access to the data collection from global 1-km nature run (NR) simulations using the Integrated Forecast System (IFS) with explicit convection. We invite you to join us in exploring this precursor to a digital twin of the earth!

The NR simulations reveal unprecedented detail of the earth’s atmosphere [1], and the then outgoing Editor-in-Chief of AGU JAMES commended the project as one of “stunning ambitions,” enabled by computational capacity at scale [2]. The project also won the 2020 HPCwire Readers Choice Award for Best Use of HPC in Physical Sciences. 

A set of two NR seasonal simulations have been completed, one corresponding to the northern hemispheric winter months (NDJF) and the other for the North Atlantic tropical cyclone season (ASO). The project used the Summit supercomputer at the Oak Ridge Leadership Computing Facility (OLCF). The simulations were facilitated with an INCITE award from the US Department of Energy Office of Science. 

For the first seasonal run of four months (NDJF), the hydrostatic IFS model was initialized at 00Z on 1 November 2018. The NR for the TC season (AS) was initialized at 00Z on 1 August 2019. The NR simulations were constrained only by sea surface temperatures (SST) at the lower boundary. The IFS output was saved every 3 hours.

After feedback and interest from the scientific community, the simulations were rerun for four specific extreme events, with output every 15 minutes. The special cases include a tropical cycle and three severe storm events over the continental USA.

NR Data for visualizing winds

A small subset from the 1-km IFS NR collection is make available for #VisMetHack22. This subset is extracted from the tropical cyclone area in the North Atlantic from the ASO simulations. The 912 model time steps correspond to 97935 to 111600 in minutes since the NR reference time 2019-08-01 00:00:00. The time increment is 15 minutes, corresponding to the output frequency.

The following variables are provided for #VisMetHack2022:
 

Data processing

1. The native model output was in the form of data objects consisting of GRIB1/2 16-bit AEC compressed messages. The messages were extracted from the FDB database instances into one or more files.

2. The files containing the GRIB messages were interpolated to 0.02 x 0.02 a regular latitude-longitude grid using ECMWF Meteorological Interpolation and Regridding (MIR), and then written out to files as GRIB messages.

3. The MIR output files were extracted to the area of interest (AOI) from global fields, and converted to Netcdf-4 (NC).

4. The metadata in NC4 files were selectively edited or added.

5. Finally, the NC4 files were compressed to reduce volume using the ncks utility from Netcdf Operators (NCO), with lossless L1 compression.

6. The variable ‘xtprate’ was calculated by a summation of instantaneous and large scape rainfall and snowfall rates.

Contact

Valentine Anantharaj <vga@ornl.gov> or <vga1.ornl@gmail.com> 

Samuel Hatfield <Samuel.Hatfield@ecmwf.int>

Citation and references

Please cite the following manuscript as well as the DOI provided by Zenodo:

[1] Wedi, N. P., Polichtchouk, I., Dueben, P., Anantharaj, V. G., Bauer, P., Boussetta, S., et al. (2020). A baseline for global weather and climate simulations at 1 km resolution. Journal of Advances in Modeling Earth Systems, 12, e2020MS002192. https://doi.org/10.1029/2020MS002192

[2] Anantharaj, V., Hatfield, S. and Vukovic, Milana (2022). VisMetHack2022: Visualizing winds and surface variables from the ECMWF IFS 1-km nature run. https://doi.org/10.5281/zenodo.6633929

Acknowledgements

This research used resources of the Oak Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC05-00OR22725.

ECMWF also  benefited from collaborations funded via ESCAPE-2 (No. 800897), MAESTRO (No. 801101), EuroEXA (No. 754337), and ESiWACE-2 (No. 823988) projects funded by the European Union's Horizon 2020 future and emerging technologies and the research and innovation programmes.