The QA Lookback Loop: Milestone Retros at Agentic Scale

Conventional postmortem discipline catches failures after they ship; it cannot catch the operational learning that fires on success cases or the drift that accumulates across many sessions. Agentic systems generate operational signal at higher cadence than conventional postmortem can process, and the signal is multi-cadence — per-event, per-session, and per-milestone patterns require different aggregation surfaces.

This paper specifies the QA Lookback Loop: a three-layer architecture with daily lookbacks (Klaus-curated, semi-automated), milestone retros (per story-close and epic-close, three-section format — Lessons Learned, Microbreakthroughs, Problem Set), and a continuously growing compendium (Klaus-gatekept, sprint-planning-readable). Each layer detects what the others cannot; together they form the operational learning surface that converts per-event signal into doctrine-update inputs. A meta-failure case study (the 2026-04-29 stale-state INFERENCE-FAILURE lookback) demonstrates the lookback-of-lookback discipline: a lookback that fails its own discipline is its own data point, and the gap-acknowledged preamble in every artifact is the cheapest possible loop-hardening response.

This is Paper Five of the Managing Agentics Ops decalogy.