Algorithmic weather model optimisation based on ensemble forecasting

Abstract

Algorithmic model optimisation is a promising approach to yield the best possible forecast performance of multi-scale multi-phase atmospheric models once the model structure is fixed. Algorithmic methods are intended to decrease the need for laborious and time consuming manual optimisation. This thesis explores different aspects related to ensemble forecasting-based optimisation of numerical weather prediction models using OpenIFS weather model as an example. This thesis firstly presents tools and initial states data to perform academic ensemble forecasting experiments. Secondly, making optimisation experiments efficiently is discussed and demonstrated. Thirdly, a potential observation-based ensemble verification method to be used in future tuning experiments is presented and discussed.

Development and testing of tools required in ensemble forecasting-based algorithmic optimisation is presented first. The tuning experimentation in this thesis requires a comprehensive set of initial states and a workflow management system that enables automatic ensemble forecasting, cost function evaluation and parameter sampling. A year of initial states for OpenIFS weather model is created. The dataset contains initial states twice daily for three model resolutions. An open-source workflow manager is presented and demonstrated with various ensemble forecasting experiments.

The forecasting tool and initial states are used to study different aspects of ensemble forecasting-based algorithmic optimisation. Firstly, parameter convergence is studied in a controlled test set-up in order to find out to what extent algorithmic optimisation can be trusted. Experimentation with the test set-up crystallises which conditions are sufficient to obtain reliable results and thus build confidence on the optimisation in fully-realistic cases. Output of the parameter convergence tests is condensed into a general guidance for algorithmic model optimisation. Secondly, the guidance is tested in fully-realistic optimisation experiments involving O(20) most important uncertain parameters of OpenIFS. It is shown that optimisation with O(20) ensemble members and O(100) algorithm steps leads to substantially improved predictive skill. However, the results show that algorithmic optimisation has a number of shortcomings too.

Lastly, so-called filter likelihood score (FLS) is introduced. FLS is an observation-based ensemble forecast verification metric that could technically be used as cost function in model optimisation in the future. FLS is capable of addressing some shortcomings encountered during the tuning experimentation in this thesis. Performance of FLS with different types of observations is demonstrated together with traditional verification metrics. The results show that FLS is capable of indicating bias and inappropriate ensemble spread.