The Risk of Negatively Biased and Over-confident Return Level Estimates: A Critique of the Metastatistical Approach to Extremes

Classical extreme value analysis (EVA) often provides large uncertainties on estimated return levels due to limited amount of data available. Marani and Ignaccolo (2015) aim to overcome this by the metastatistical extreme value (MEV) approach. Here extremes are treated as large ordinary events described by one common, known distribution, and therefore a much larger pool of data are available for estimation. They performed Monte Carlo simulations with synthetic Weibull-distributed rainfall series and showed that the MEV approach gives unbiased estimates of extremes with a smaller uncertainty than classical EVA does. However, the MEV approach neglects that many complex physical mechanisms influence rainfall. This means that the tail behavior of the distribution cannot be inferred from the ordinary events. We therefore replicated their work but added new Monte Carlo experiments to study the classical EVA and the MEV methodologies with a slightly perturbed tail of the underlying distribution. When applying the MEV approach, i.e. fitting a Weibull distribution to the perturbed Weibull series, we obtained systematically negatively biased estimates with too narrow confidence intervals – MEV became over-confident. In contrast, classical EVA also here produced unbiased estimates. Finally, we showed that goodness-of-fit tests are not able to provide guidance on whether MEV can provide unbiased and confident return levels. Further Monte Carlo simulations showed that these conclusions seem to be quite general and not dependent on the specific distribution. Consequently, the MEV approach is unsuitable to provide reliable return levels, and we strongly caution against using it in real-world applications.