Published October 16, 2026 | Version v1.1.1

brightbandtech/ExtremeWeatherBench: v1.1.1

  • 1. @TheWeatherCompany
  • 2. @Brightbandtech
  • 3. @fraunhoferhhi

Description

Added

  • defaults.preprocess_derive_geopotential: a forecast preprocess that derives geopotential for forecasts that don't publish it, using the new calc.geopotential_from_temperature_and_humidity (hypsometric integration of temperature and specific humidity upward from mean sea level pressure, a Numba column kernel applied lazily). Lets tropical cyclone tracking and the Craven-Brooks significant severe parameter (CBSS) run on such forecasts; a no-op when geopotential or geopotential_height is present (#433).
  • station_qc module (ewb.station_qc) with the GHCNh checks used by the GHCN target (exclude_stations, mask_sentinel_values, mask_out_of_range, mask_spikes, apply_ghcnh_row_checks, apply_ghcnh_quality_control) and their default settings (GHCNH_* constants), plus a GHCN.quality_control field (default True) (#431).
  • Two InputBase hooks for custom inputs, both no-ops by default: maybe_apply_quality_control(data), which run_pipeline calls right after variable mapping and before preprocess, and columns_kept_through_variable_subset(), which names extra columns to keep through the tabular variable subset. maybe_subset_variables takes a matching keep_if_present= argument (#431, #435).

Changed

  • Clear any cache_dir reused from 1.1.0 runs. Cached aligned targets are named by case and target name only, so a cache written by 1.1.0 is reused as-is and its GHCN targets skip the new quality control.
  • GHCN drops rows with no non-null values before building the dataset, and never turns the station or name columns into data variables, whether or not quality_control is on. Hours whose only value was nulled are therefore missing from the time axis rather than NaN, and a valid row now wins over an empty duplicate at the same time and place, so quality_control=False does not exactly reproduce 1.1.0.
  • GHCN quality control expects raw GHCNh units (°C, hPa, m/s, %) under EWB column names; it runs after variable_mapping, so sources with other column names are checked too. A custom GHCN-format source in other units (e.g. Kelvin temperatures) must pass quality_control=False, or the range check nulls its values. With quality control on, subclasses that override GHCN._custom_convert_to_dataset also receive the station column, and should drop it (#435).

Fixed

  • The GHCN target now quality-controls station observations before scoring (new station_qc module, on by default; pass quality_control=False to turn it off). It nulls missing-value sentinels such as -999 (e.g. one station reporting -999 C for all of 2020), physically impossible values (e.g. 928 C air temperature), and isolated up-then-down spikes in temperature and dew point. The spike check can also remove a real short-lived extreme such as a heat burst; pressure is not spike-checked, since tropical-cyclone eye passages and derecho wake lows look like spikes (pressure still gets the sentinel and range checks). It also drops the 39 Greenland ice-sheet stations named "MOVING ICE STATION", several of which report temperatures far too warm for weeks at a time (e.g. NEM averages +4.3 C over Jan-Mar 2023); this is a stopgap until a climatology or neighbor-station check exists. GHCN scores for cases touching affected stations change, and a case whose only stations are excluded is no longer scored against GHCN (#431).
  • GHCN and LSR now subset correctly for case regions that cross the antimeridian. They filtered on the region's total_bounds, which for a crossing region is -180..180, so the whole latitude band was kept (e.g. a 166.5E-190E box took stations from Alaska to Scandinavia). They now use the same wrapped bounds as the gridded targets. A region spanning 359 degrees of longitude or more (e.g. the 0E-359.75E boxes of cases 1000-1003) is still treated as global, rather than losing the stations just west of 0. LSR also no longer drops a region whose east edge is exactly 180, which was converted to -180 (this also makes LSR select the near-global boxes, which it previously left empty). Default (events.yaml) scores are unchanged; GHCN and LSR scores for the affected marginal-temperature and long-event cases change (#432).

Full changelog: https://github.com/brightbandtech/ExtremeWeatherBench/compare/v1.1.0...v1.1.1

Files

brightbandtech/ExtremeWeatherBench-v1.1.1.zip

Files (957.9 kB)

Name Size
md5:db41c5397405c215696a8995629b23a5
957.9 kB Preview Download

Additional details

Related works