There is a newer version of the record available.

Published September 23, 2026 | Version v1.3

Where Foundation Machine-Learning Potentials Fail for Iron-Sulfide Defect Kinetics: Failure Modes, Pre-Flight Checks, and a DFT Benchmark of Vacancy-Anchored Hydrogen Migration in Pyrite, Marcasite, Mackinawite, and Greigite

Authors/Creators

  • 1. Independent Researcher, Ukraine

Description

The fine-tuning experiment requested by referee 2 is deposited: whether limited fine-tuning recovers a DFT barrier the model has not seen, and therefore whether the zero-shot failure reported in this work is a failure of data coverage or of the approach.

The ladder. mlip/r23_2026-09/ fine-tunes MACE-MP-0 large — the same checkpoint benchmarked in the paper — on the DFT bands already deposited here, holding one mineral out and increasing what the model is told about it rung by rung: nothing; the four bands of the three other iron sulfides; those plus both endpoint basins of the held-out mineral; those plus its full band; and its band alone. Eight seeds per rung, with the validation split identical across rungs at a given seed, so the comparison is paired. The discriminator is the paired difference between the third-mineral rung and the in-domain rung, not "did the third-mineral rung recover" — the in-domain rung is deliberately tautological and works as a ceiling rather than as a result. A fifth rung trained on the held-out band alone was added so that a negative in-domain result could be told apart from conflicting supervision between minerals, which here pull in opposite directions by +149, +159, −15 and −132 meV.

Result, mackinawite held out (DFT 42.88 meV). Barrier error against DFT, mean ± s.d. over eight seeds, single points on the fixed DFT band: +149.1 zero-shot → +68.3 ± 22.7 after fine-tuning on the three other sulfides → +68.6 ± 17.4 with both endpoint basins added → +9.1 ± 7.1 with the held-out band added → +0.5 ± 5.9 trained on that band alone. Paired over seeds the difference is 59.3 meV, 95 % CI [36.7, 81.8], exact sign-permutation p = 0.0078 — the smallest value eight pairs can produce. The same ordering holds when each fine-tuned model runs its own NEB with its own relaxed endpoints.

So the zero-shot failure is one of data coverage, not of the approach: the architecture reproduces this barrier to sub-meV once it has seen the reaction coordinate of that mineral. What does not happen is transfer between iron sulfides. And supplying both endpoint basins changes nothing, because what is missing is the saddle region — which is exactly what cannot be had without the DFT one hoped to avoid.

Two results that do not flatter the experiment, reported rather than buried.

The ladders were trained with a schedule that fails the gate declared before the runs: at 200 epochs the best configuration reproduces the training bands' own barriers only to 31.6 meV against a 10 meV threshold. The same hyperparameters run to convergence pass at 5.4 meV, so the third-mineral rung was re-run to convergence: a twentyfold longer schedule leaves the held-out error at 51.6 ± 18.8 meV, far outside the 25 meV recovery tolerance, so the conclusion is not an artefact of training length. In the self-consistent convention that longer schedule instead reverses the sign of the error (−25.4 ± 18.3 meV) and inflates the seed-to-seed spread to 47 meV — larger than the 42.88 meV barrier being predicted — with the barrier vanishing entirely in one seed of eight. control_schedule_length.json carries all of it.

A second ladder with marcasite held out is deposited and explicitly not interpreted. Its in-domain rung fails to reproduce the band it was trained on (−56.1 ± 22.7 meV, against +0.5 for mackinawite), which is the same gate failure — 200 epochs suffices for a 42.88 meV barrier and not for a 208 meV one — and its discriminator is flat (p = 0.625). Its one usable number is that zero-shot MACE-MP-0 reproduces the marcasite barrier to −1.44 meV in the self-consistent convention, the closest agreement anywhere in this work. That is why marcasite is a poor holdout for a recovery experiment, and why the holdout is mackinawite, where all nine benchmarked models overestimate the barrier and MACE-MP-0 does so by 4.5×.

How little data this actually is, stated rather than rounded up. The training pool is 35 band images, but one pair is a bit-identical duplicate, and both greigite bands are exactly mirror-symmetric — images i and 8−i are isometric to 1.3·10⁻⁵–8.4·10⁻⁵ Å in their interatomic-distance spectra, and MACE reproduces the pairs to 0.07–0.10 meV, i.e. numerical noise. For an E(3)-equivariant model the second half of each greigite band carries no information. That leaves 26 symmetry-inequivalent configurations, and an effective n of about four paths, since nine images of a band are points on one trajectory. Training uses the 26.

Blind schedule selection. schedule_sweep.json records five configurations, the gate, and the outcome. The selection script never opens the holdout band; among configurations that pass the gate the choice is made on validation force RMSE, an out-of-sample quantity. The holdout was read only after the schedule was fixed.

Also deposited: per-seed dataset specifications naming exactly which band image sits in training and which in validation for every rung and seed; per-model evaluations with profiles, barriers, reaction energies and per-image errors; the 78 MACE training logs that carry the convergence traces behind the schedule argument; and the eight pipeline scripts, from the zero-shot baseline through dataset construction, verification, training, evaluation and aggregation.

Model weights are deliberately not deposited — 77 checkpoints, 1.7 GB, reproducible from the deposited bands, dataset specifications, seeds and scripts, with the caveat that MACE training on GPU is not bit-reproducible and barriers will differ at the few-meV level. Optimiser states and duplicate checkpoint copies, a further 8.8 GB, carry no scientific information and are omitted.

A reproducibility fix. scripts/per_image_stats.py and mlip/q5_correlation.py carried a hard-coded local filesystem path from earlier releases. Both now take their paths from the environment and run from a clone without editing.

Citing. Releases from v1.1 onward carry corrections that change published numbers. Cite the release that matches the numbers you use. Retracted material remains in the repository, flagged, so that anyone who cited an earlier release can still resolve what they cited.

Notes

If you use this code or data, please cite the accompanying paper.

Files

exopoiesis/mlip-vs-dft-iron-sulfides-v1.3.zip

Files (1.6 MB)

Name Size
md5:0885a3f93290903e2e576eb6131b65ca
1.6 MB Preview Download

Additional details

Related works