Self-Obtained and Evidence-Driven, but Not Sustained: Two Preregistered Level-4 Tests of a Self-Editing Campaign Layer

FELIX-C9 was the first study in this programme in which rewriting a program-repair system's improvement mechanism (Imp) paid off causally. Its gain rested on one of two campaigns. Here FELIX-C9's mission code is kept unchanged, and its campaign layer is replaced by a self-editing one. That layer keeps an archive of variants, has an improver program that edits itself, has writable parent-selection rules, and adopts a change only through a fixed gate outside anything it can edit.

In FRM-11 (preregistered; 2 campaigns, then 72 confirmatory missions on 24 fresh seeds), both campaigns independently found and committed the same structural rewrite, a tree-ensemble policy form, on their own hidden-seed evidence. The evolved Imp beat the default by +4.71 pp (95% CI 3.85 to 5.56; 23 of 24 seeds; exact one-sided Wilcoxon p = 1.2 × 10⁻⁷). FRM-12 (preregistered; confirm-only, 72 missions on 24 further fresh seeds, wider per-round search) replicated this at +4.92 pp (24/24, p = 6 × 10⁻⁸). It also showed that the choice was driven by evidence. A per-seed random pick from the campaigns' own proposals did no better than the default (+0.23 pp), and the evolved Imp beat it by +4.69 pp (22/0/2, p = 2.4 × 10⁻⁷).

Both studies nonetheless failed the ladder's sustained-progress criterion A-4, which requires a gain of at least 1 pp from round 16 to round 32 at p < 0.01: +1.08 pp (p = 0.026) and +0.81 pp (p = 0.021). Their preregistered verdict is therefore "B-1 pass but Tier A fails", not "Level 4 with causal support". The failure is structural. The evolved Imp reaches about 45% by round 16 and has little left to gain, while the far weaker default passes A-4 in both studies (p = 0.003 and 5 × 10⁻⁶). No gain reached held-out task families. We also report a defect in FRM-11's random control, which collapsed onto the default, and how FRM-12 repaired it.

Preprint, not peer reviewed. Preregistrations, freeze records, per-seed data and a script that recomputes every reported statistic: https://github.com/euroized/FRMDevelopment