A Metric-Adaptive Hybrid Pipeline for Chaotic Lorenz Dynamics in the Common Task Framework for Scientific Machine Learning
Authors/Creators
Description
The Lorenz benchmark of the Common Task Framework (CTF) for scientific machine learning evaluates a method across twelve metrics spanning short-time forecasting, long-time statistical ("climate") forecasting, and state reconstruction, under heterogeneous data regimes: abundant clean trajectories, noise-corrupted observations, few-shot windows of 100 points, and parametric generalization. We show that this task structure is deliberately hostile to one-size-fits-all approaches: a uniformly applied Echo State Network achieves a mean score of only 35.2 on the official hidden test set. We present a metric-adaptive hybrid pipeline that routes each train/test pair to a physics- or statistics-motivated strategy—least-squares recovery of the governing equations with Runge–Kutta integration for forecastable windows, extended Kalman smoothing with the fitted dynamics as process model for reconstruction, invariant-measure quantile matching for long-time statistics, reservoir computing for clean few-shot forecasting, and a symmetry-derived distributional prior for a structurally unforecastable noisy few-shot window. The pipeline is deterministic, runs in seconds on a single CPU, and achieves a mean score of 84.98 on the official hidden test set—a +141% improvement over the uniform reservoir baseline and above the top entry of the final competition leaderboard (83.86), although our evaluation was obtained through a post-deadline submission and is therefore not part of the official ranking. The pipeline's structure was not designed top-down: it was uncovered through offline evolutionary exploration with the inZOR-ND engine, whose failure diagnostics—rather than its tuned parameters—exposed the benchmark's regime structure and reframed the hardest task as a distributional rather than a forecasting problem. Beyond the score, we document two methodological findings that the benchmark's hidden-test design makes visible: development-set tuning systematically fails to transfer to held-out trajectories, and recognizing which metric a task actually evaluates matters more than model capacity.
Notes
Files
CTF_Lorenz_Metric_Adaptive_Hybrid_Novic.pdf
Files
(161.2 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:e1826bc161b2b0b9fcf25384f2edfca5
|
161.2 kB | Preview Download |
Additional details
Related works
- Is supplemented by
- https://www.kaggle.com/competitions/ctf-4-science-lorenz (URL)
- https://github.com/CTF-for-Science/ctf4science (URL)