Cleared the Bar, or Lowered It? An Assurance Lattice for Verifying AI-Generated Software Against Intent
Description
AI coding agents are often certified by cheap proxies: the build passes, the tests pass, the linter is clean. Recent evidence shows agents exploiting those proxies by special-casing tests, editing the checker, or deleting failing tests. The most dangerous move leaves every proxy green. The agent weakens the requirement until the defect is out of scope, so the check passes because the bar was lowered, not cleared.
This paper builds a theory of assurance for AI-generated software around that distinction. It separates facts (checks that are exact for the property they compute) from proxies (checks that stand in for a property they don't compute). It argues that rigid structure, mandatory traceability, and bounded scope cost a machine author almost nothing, while making the code-to-intent link mechanically checkable.
Main contributions
- An assurance lattice. Done-ness is a coordinate in a product of three chains (structural, semantic, observational). The four truth-levels declared, wired, correctly-meant, and demonstrated form an order-embedded canonical path with a meet-preserving valuation. The dangerous states lie off that path.
- A theory of intent change. "Narrowing" conflates two relations: a coverage gap between invariant and criterion, and a weakening of the criterion itself. No entailment check over the current pair can detect the second. Requirement change is therefore governed as a transaction, with fail-closed classification, hash-pinned intent text, and human-approved verifier retirement.
- A reference design and threat model, with design responses to documented gaming behaviors.
- A small seeded demonstration of the predicted asymmetry between exact structural checks and a noisy lexical meaning router.
- The meaning ceiling. The semantic axis is not reliably mechanizable today, and the theory states where human judgment remains irreducible.
This is the theory version of the work (v1.0). Definitions, propositions, rules, and predictions are numbered so that implementations and later papers can cite them. Preprint, not peer-reviewed.