Carbon-Aware GitOps Gates Revisited: A Preregistered Trace-Driven Evaluation of CarbonGate on Great Britain Grid Data

A GitOps pipeline starts a deployment as soon as a change merges, whatever the carbon intensity of the grid at that moment. CarbonGate puts a carbon-aware admission gate in front of the GitOps reconciler. Urgent changes pass at once. Other deployments may wait for a low-intensity slot in the forecast, within a maximum-delay service-level agreement. This revision replaces the unsupported production figures of version 1 with a preregistered, trace-driven evaluation. We replay a simulated deployment workload against 12 months of half-hourly Great Britain grid data from the NESO Carbon Intensity API. The gate decides from the published forecast alone, and emissions are scored on the reported actual intensity. Over 11 held-out months and 5 seeds, forecast-based deferral with a 4 h limit cut the average-intensity emissions of deployments by 4.80% (95% CI 4.17 to 5.43%), and no deployment missed its deadline. That interval contains the preregistered 5% bound, so the primary hypothesis is inconclusive. The version 1 claim of at least 23% is refuted. A perfect-foresight oracle reached 7.33%. A static intensity threshold, which was the version 1 gate design, did not reduce emissions at 2 h or 4 h limits. Gains shrank as forecast error grew and were gone at a noise level of 50 gCO2/kWh. Under injected forecaster outages, fail-closed handling caused deadline violations; a cached persistence fallback still kept 4.24% at 50% outage. At 100 times the load, with a per-slot capacity cap, herding removed the benefit. We did not study developer acceptance.

Changes in version 2.0

• All v1 production-evaluation figures are withdrawn (deployment counts, 23-31% reductions, latency, 92% developer acceptance, false-positive and cost figures). No data or code supported them.
• In their place is a preregistered trace-driven replay on 12 months of real GB grid data (NESO Carbon Intensity API, CC BY 4.0) with a labelled simulated workload, 11 held-out months, 5 seeds and 95% confidence intervals.
• CarbonGate now has a formal specification: system model, objective, constraints, and the forecast-deferral and threshold rules. Fail-open, fail-closed and cached-persistence behaviour for forecaster outages and timeouts is defined and tested under injected failures.
• New ablations vary the forecast source and noise, delay limit, margin, urgent share, arrival shape and region. A 100x load test adds a capacity cap.
• Results as measured: the primary hypothesis (at least 5% reduction) is inconclusive at 4.80% (95% CI 4.17 to 5.43%), and the v1 claim of at least 23% is refuted. The v1 static threshold gate did not reduce emissions at 2 h or 4 h.
• Unverifiable references are replaced by 21 DOI/arXiv-verified references plus software/API documentation. The record ships the code, cached data, results, tests and preregistration.

To reproduce, unzip the code-data archive and run python code/run_all.py (Python 3.12; see code/requirements.txt).