OlssonF/Forecast-evaluation-EFI25: EFI 2025 - Forecast Evaluation workshop

Materials for the Forecast Evaluation workshop produced for and presented at the Ecological Forecasting Initiative Conference 2025 by Freya Olsson, Caleb J. Robbins, and R. Quinn Thomas. These materials contain a tutorial that introduces concepts of ecological forecast evaluation and synthesis.

The tutorial is structured as follows:

1. Introduction to scoring and evaluation: We include the presentation given during the in-person workshop that provides a lot of the background and rationale for the evaluation of ecological forecasts and the use of large forecast catalogs for forecast synthesis.
2. Accessing NEON catalog: This section is a quick introduction to the EFI-NEON Forecast Challenge catalog, which will be used to demonstrate forecast evaluation themes. A more in-depth look at the NEON forecasts from the Challenge can be found in the Get_scores_tutorial as part of the NEON Forecast Challenge Workshop.
3. Calculating and plotting evaluation metrics: A demonstration in R showing the plotting of forecast scores for single and multi-model comparisons.
4. Further considerations: An open discussion on the practical considerations for conducting large-scale synthesis efforts using forecast catalogs (e.g. unequal forecasts, metadata, co-authorship).
5. References and further reading: Key references and places to get a more in-depth understanding of forecast scoring and evaluation relevant to this workshop.

These materials were developed with funding support from National Science Foundation grant DEB-1926388 (The Ecological Forecasting Initiative RCN: Using NEON-enabled near-term forecasting to synthesize our understanding of predictability across ecological systems and scales).