Report the Floor: A Training-Free Conformal Interval Is a Mandatory Baseline for Probabilistic Time-Series Forecasting
Authors/Creators
Description
Probabilistic forecasters are increasingly learned, yet the baselines they are compared
against are often weak or omitted. We show that the simplest possible conformal interval
— a last-value point forecast wrapped in a finite-sample split-conformal residual quantile,
with no parameters and no training — is a far stronger baseline than its near-total absence
from recent learned-forecasting and conformal–time-series comparisons would suggest.
In one-step-ahead online forecasting across 2,217 real series spanning nine public sources
(the Monash archive, the LOTSA collection, the LTSF traffic/electricity/weather suites,
METR-LA, BOOM, and nips/probts), this ConformalNaive interval decisively beats the
naive value-quantile baselines, the entire NPTS family (NPTS 73%, SeasonalNPTS 64% of
series), and the published Conformal Seasonal Pools (CSP) method (71% of series, bootstrap
95% CI [69, 73], p ≈ 7.6 °ø 10−135); it is on par with the simpler learned conformal
predictors (RCI, quantile regression; median relative Winkler within 2%) and is beaten only
by the adaptive-online and ensemble conformal methods (SPCI, ACI, AgACI), which explicitly
track distribution shift and lead by 9–33% relative Winkler. It is also better calibrated
than a trained neural forecaster: on the six datasets that introduced DeepNPTS, the trivial
conformal floors cover the truth 84–85% of the time at a nominal 95%, versus DeepNPTS’s
66%. At multi-step seasonal horizons the picture inverts: the random-walk floor is the
weakest method and the seasonal pool (CSP) wins — a boundary we map so practitioners
know when complexity is actually required. Finally we give ConformalNaive+, a one-line,
training-free, horizon-adaptive selector that attains the better of two complementary floors
at every horizon with restored coverage. We argue the matching conformal naive floor must
be a mandatory baseline whenever a learned probabilistic forecaster claims gains.
Files
main.pdf
Files
(359.1 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:5407ac7764e5b07cbeb9f886452ff823
|
359.1 kB | Preview Download |