Data fusion technology - kriging (TURBAN - D11)

Abstract

This package is the deliverable D11 in WP4 of the TURBAN project. The package consists of software (python scripts) and a demonstration example of usage.  The aim is to obtain statistically optimal meteorological map on a regular spatial grid of high resolution.

Data

The package scripts combine and interpolate provided heterogenioes datasets of high resolution (NETATMO data and PALM simulations). The software package accepts diverse meteorological data sets, namely, the meteorological observations from stations (here given by the NETATMO set of stations from the Bergen data collected for theproject, see Esau, 2023) and results of PALM simulations (here, the PALM run in large domain set2, see Esau et al., 2024). The package also utilized the digital elevation model (DEM) of the Norwegian Mapping Authorities profived in geo-tiff format.

Method

The package uses the methods of ordinary and universal kriging with the model external drift. This set of methods were found optimal in several published tests, e.g., Cowtan and Way (2014), Varentsov et al. (2020), Kadow et al. (2020). The study in the TURBAN project (in preparation for publication) suggested that additional complexity introduced in Varentsov et al. (2020) becomes redundant when PALM simulations are used for variogram calculation and externa drift. Since PALM output automatically account for data dependence on elevation, DEM model becomes redundant too. The selection of variogram models is automated and optimized.

The data fusion scripts use the standard (Pandas, Xarray, Matplotlib, Cartopy, Numpy, Json, Os, Datetime) and the following specialized python packages:

• OSM_map - adopted script to plot the urban-scale basemap (from M. Lipton with significant modifications)
• Gstools - the python package that provides geostatistical tools for various purposes, including variogram calculation and krigging (https://geostat-framework.readthedocs.io/projects/gstools/en/stable/, license LGPLv3) by Müller et al. (2022)

How to use

The data fusion scripts are run by main_kriging_2024v04.py from the folder Data_fusion_scripts.

Before running the kriging script, a case must be created by setup_case.py script. This script sets the data path, data types (NETATMO, PALM, DEM), and read input data from the raw data supplied (scripts *_preparation.py). Because (NETATMO, PALM, DEM) are archetypical types of data:

• NETATMO - for station observations - irregular pointwise data type; could be used WMO data, shadow meteo-data, etc.
• PALM - for model output - regular gridded data type; could be used weather prediction, air quality model, or gridded observation data.
• DEM - for satellite images - geo-projected images data type; could be substituted/expanded with other similar data sources.

References

Ezau, Igor, 2023, "TURBAN – Observational datasets for studies of urban air quality hazard scenarios in Bergen, Norway", https://doi.org/10.18710/QHUAZ2, 

Esau, I., Miles, V., Bures, M., Resler, J., & Eben, K. (2024). Scenarios simulations of Bergen (TURBAN - D06) (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.11045535

Cowtan, K., Way, R.G., 2014. Coverage bias in the HadCRUT4 temperature series and its impact on recent temperature trends. Q. J. R. Meteorol. Soc. 140, 1935–1944. https://doi.org/10.1002/qj.2297

Kadow, C., Hall, D.M., Ulbrich, U., 2020. Artificial intelligence reconstructs missing climate information. Nat. Geosci. 13, 408–413. https://doi.org/10.1038/s41561-020-0582-5

Müller, S., Schüler, L., Zech, A., Heße, F., 2022. GSTools v1.3: a toolbox for geostatistical modelling in Python. Geosci. Model Dev. 15, 3161–3182. https://doi.org/10.5194/gmd-15-3161-2022

Varentsov, M., Esau, I., Wolf, T., 2020. High-Resolution Temperature Mapping by Geostatistical Kriging with External Drift from Large-Eddy Simulations. Mon. Weather Rev. 148, 1029–1048. https://doi.org/10.1175/MWR-D-19-0196.1