xclim: xarray-based climate data analytics

Contributors to this version: Adrien Lamarche (@LamAdr), Trevor James Smith (@Zeitsperre), Éric Dupuis (@coxipi), Pascal Bourgault (@aulemahal), Sascha Hofmann (@saschahofmann), David Huard (@huard).

Announcements

• xclim has now adopted the Scientific Python SPEC 0 conventions for its suggested dependency support schedule. (GH/1914, PR/1915).
• xclim has dropped support for Python 3.9 and adopted Python 3.10+ code styling conventions. (GH/1914, PR/1915).

New indicators

• New heat_spell_frequency, heat_spell_max_length and heat_spell_total_length : spell length statistics on a bivariate condition that uses the average over a window by default. (PR/1885, PR/1778).
• New hot_spell_max_magnitude : yields the magnitude of the most intensive heat wave. (PR/1926).
• New chill_portion and chill_unit : chill portion based on the Dynamic Model and chill unit based on the Utah model indicators. (GH/1753, PR/1909).
• New water_cycle_intensity : yields the sum of precipitation and actual evapotranspiration. (GH/410, PR/1947).

New features and enhancements

• New generic xclim.indices.generic.spell_mask that returns a mask of which days are part of a spell. Supports multivariate conditions and weights. Used in new generic index xclim.indices.generic.bivariate_spell_length_statistics that extends spell_length_statistics to two variables. (PR/1885).
• Indicator parameters can now be assigned a new name, different from the argument name in the compute function. (PR/1885).
• Add attribute units_metadata to outputs representing a difference between temperatures. This is needed to disambiguate temperature differences from absolute temperature. Changes affect indicators daily_temperature_range, daily_temperature_range_variability, extreme_temperature_range, interday_diurnal_temperature_range, and all degree-day indicators. Implemented using a new pint2cfattrs function to convert pint units to a dictionary of CF attributes. units2pint is also modified to support units_metadata attributes in DataArrays. Some SDBA properties and measures previously returning units of delta_degC will now return the original input DataArray units accompanied with the units_metadata attribute. (GH/1822, PR/1830).
• xclim.indices.run_length.windowed_max_run_sum accumulates positive values across runs and yields the maximum valued run. (PR/1926).
• Helper function xclim.indices.helpers.make_hourly_temperature to estimate hourly temperatures from daily min and max temperatures. (PR/1909).
• New global option resample_map_blocks to wrap all resample().map() code inside a xr.map_blocks to lower the number of dask tasks. Uses utility xclim.indices.helpers.resample_map and requires flox to ensure the chunking allows such block-mapping. Defaults to False. (PR/1848).
• xclim.indices.run_length.runs_with_holes allows to input a condition that must be met for a run to start and a second condition that must be met for the run to stop. (PR/1778).
• New generic compute function xclim.indices.generic.thresholded_events that finds events based on a threshold condition and returns basic stats for each. See also: xclim.indices.run_length.find_events. (PR/1778).
• xclim.core.units.rate2amount and xclim.core.units.amount2rate can now also accept quantities (pint objects or strings), in which case the dim argument must be the time coordinate through which we can find the sampling rate. (PR/1778).
• xclim.indices.stats.standardized_index now supports a weekly resampling frequency. Only "standard" calendars using numpy's datetime64 dtype are supported for this mode. (GH/1892, PR/1952)

Bug fixes

• Fixed rate2amount and amount2rate for sub-daily frequencies. (GH/1962, PR/1963).
• Added the liquid water equivalent thickness ("[length]") to amount ("[mass]/[area]") transformation to the hydro context (the inverse operation was already there). (PR/1963).
• Fixed a small inefficiency in _otc_adjust, and the standardize method of OTC/dOTC is now applied on individual variable. (PR/1890, PR/1896).
• Removed deprecated cells in the tutorial notebook sdba.ipynb. (PR/1895).

Breaking changes

• platformdirs is no longer a direct dependency of xclim, but pooch is required to use many of the new testing functions (installable via pip install pooch or pip install 'xclim[dev]'). (PR/1889).
• The following previously-deprecated functions have now been removed from xclim : xclim.core.calendar.convert_calendar, xclim.core.calendar.date_range, xclim.core.calendar.date_range_like, xclim.core.calendar.interp_calendar, xclim.core.calendar.days_in_year, xclim.core.calendar.datetime_to_decimal_year. For guidance on how to migrate to alternatives, see the version 0.50.0 Breaking changes. (GH/1010, PR/1845).
• The transform argument of OTC/dOTC classes (and child functions) has been changed to normalization, and numIterMax has been changed to num_iter_max in xclim.core.utils.optimal_transport (PR/1896).
• xclim now requires numpy >=1.23.0 and scikit-learn >=1.1.0, as well as (optionally) ipython >=8.5.0, nbsphinx >=0.9.5, and matplotlib >=3.6.0. (GH/1914, PR/1915).

Internal changes

• The Ouranosinc/xclim-testdata repository has been restructured for better organization and to make better use of pooch and data registries for testing data fetching (see: xclim-testdata PR/29). (PR/1889).
• The xclim.testing module has been refactored to make use of pooch with file registries. Several testing functions have been removed as a result: (PR/1889)
  • xclim.testing.utils.open_dataset now uses a pooch instance to deliver locally-stored datasets. Its call signature has also changed.
  • xclim now accepts more environment variables to control the behaviour of the testing setup functions. These include XCLIM_TESTDATA_BRANCH, XCLIM_TESTDATA_REPO_URL, and XCLIM_TESTDATA_CACHE_DIR.
  • xclim.testing.utils.get_file, xclim.testing.utils.get_local_testdata, xclim.testing.utils.list_datasets, and xclim.testing.utils.file_md5_checksum have been removed.
    • xclim.testing.utils.nimbus replaces much of this functionality. See the xclim documentation for more information.
• Many tests focused on evaluating the normal operation of remote file access tools under xclim.testing have been removed. (PR/1889).
• Setup and teardown functions that were found under tests/conftest.py have been optimised to reduce redundant calls when running pytest xclim. Some obsolete pytest fixtures have also been removed. (PR/1889).
• Many DeprecationWarning and FutureWarning messages emitted from xarray and pint have been addressed. (GH/1719, PR/1881).
• The code base has been adjusted to address many pylint-related warnings and errors. In some cases, casting was used to redefine some numpy and xarray objects. (GH/1719, PR/1881).
• xclim.core now uses absolute imports for clarity and some objects commonly used in the module have been moved to hidden submodules. (GH/1719, PR/1881).
• xclim.core.indicator.Parameter has a new attribute compute_name while xclim.core.indicator.Indicator lost its _variable_mapping. The translation from parameter (and variable) names in the indicator to the names on the compute function is now handled by Indicator._get_compute_args. (PR/1885).
• Adopted many linting and formatting suggestions from the Scientific Python repo-review tool: (PR/1910)
  • Applied several linting suggestions adopted by the scipy community.
  • Replaced isort with ruff-based import-sorting formatting.
  • Added formatting for Markdown files.
  • Added the bugbear, pyupgrade checks to the ruff formatter.
  • Adjusted mypy checks to be more standardized.
• Renamed annual deprecated frequency alias "A" to "Y" (PR/1930).
• The indices documentation now includes the members of xclim.indices.stats. (GH/1913, PR/1958).
• The default URL for fetching testing data is now set to the raw.githubusercontent.com mirror of xclim-testdata. (PR/1961).
• The upstream tox environment has been updated to not install the latest numpy until numba supports it. (PR/1961).

CI changes

• The pip cache, tox environments, and the xclim-testdata cache are now saved between workflow runs (using actions/cache) to reduce the time spent installing dependencies and downloading testing data. (PR/1906).