Published October 17, 2026 | Version v1
Preprint Restricted

Does the Train-Gain/Generalization Relation Transfer? A Pre-Registered Cross-Domain Test of Inverted-U Behavior in Verifier-Bound Rule Discovery

  • 1. Shenyang Institute of Technology

Description

A verifier-bound rule-discovery engine –- an LLM proposes executable rules, a programmatic verifier gates them, a single-shot isolated holdout scores them –- was run in three domains under one pre-registered methodology. The first domain (combinatorial scheduling) reported an inverted-U relation between training improvement and holdout generalization: moderate training gains generalized best, and the largest gains overfit. This paper reports the pre-registered cross-domain test of that relation. Under the frozen protocol (quartile buckets over margin-positive pairs; criterion: middle buckets' mean win rate exceeds the ends' by 0.05; $ 2$ of $ 2$ participating domains required), the criterion replicates: scheduling $0.35/0.60/0.70/0.48$ (mid $0.650$ vs.\ ends $0.413$) and MCS task assignment $0.36/0.93/1.00/1.00$ (mid $0.964$ vs.\ ends $0.679$). The second domain (MCS incentive design) is excluded by the protocol's own rule –- its verifier passed 3 of 2,000 LLM rules and 0 of the human-designed references, a threshold effect that leaves no margin-positive pairs. The substantive finding is not the verdict but the shape divergence beneath it: in scheduling the top quartile declines (48\ overfitting), while in assignment the top quartile saturates at 100\ different mechanisms. We read the pattern –- hard 0–1 verification collapses generalization, mid-severity simulation lets the top quartile overfit, feasibility-by-construction transfers it –- as evidence that verifier strictness determines the fate of the largest training gains, and we report the two-level verdict (criterion: replicated; mechanism: not shared) as the result. Pre-registrations, ledgers, and analysis scripts are available from the authors upon reasonable request via email.

Files

Restricted

The record is publicly accessible, but files are restricted. Log in to check if you have access.