Published October 28, 2026 | Version v1

Which decisions will the next model change? A label-free budget and watchlist for AI model updates, tested on a live version change

Authors/Creators

  • 1. Bulb's Head SAS; Université Paris Dauphine-PSL

Description

A system that reads its decisions off a model's probabilities—a classifier, a language-model judge, a typed-decision model—inherits a problem its vendor does not solve: the model behind the identifier changes, and nobody can say in advance which past decisions the next version will reverse. When the decision is compiled from a rule over thresholded answers, the answer is in its margins, computed label-free from the snapshot: the margin of a decision is the smallest move of the answers that flips a contrastive explanation of it. It gives a budget—an update whose answers drift by at most $\delta$ changes at most $\#\{\text{margin} \le \delta\}$ decisions, published before the update exists—and a watchlist of the decisions to re-examine first. We tested both, preregistered, on a hosted model's live version change, measured at the release and again at the alias switch nine days later: on 300 documents the compiled rule changed 0 decisions both times where a single reading question changed 25 and 29 and a whole-decision question 24 and 25; the margin ranked the changes with AUROC 0.86–0.98, a 20% watchlist held 67–100% of them, and a certificate restricted to the literals the rule reads budgets 6 of 200 decisions against 71 and 138. After the update, a replayed sentinel's change rate bounds the number of changes more tightly; the margin's value is before, and in saying which.

Files

Chahed_2026_which-decisions-next-model_preprint.pdf

Files (622.1 kB)

Name Size
md5:9aef4559fe50c86779df10207bdc28d3
622.1 kB Preview Download