Published April 7, 2026 | Version v1
Preprint Open

The Intervention Penalty: A Simulation Study of Human Checkpoint Costs in AI Coding Governance

Authors/Creators

  • 1. bosun.sh

Description

This paper examines whether human-in-the-loop (HITL) checkpoints—common in AI agent deployments—actually improve or degrade performance. The authors hypothesize that frequent human intervention imposes a measurable "intervention penalty" on AI agents, similar to how micromanagement affects human workers. They formalize this penalty mathematically and explore it through simulation, finding that intervention frequency may be a stronger predictor of task degradation than residual capability variation. The paper argues for "structured self-governance" over action-level oversight in software development contexts where AI capability is sufficient.

Files

The Intervention Penalty: A Simulation Study of Human Checkpoint Costs in AI Coding Governance.pdf

Files (1.0 MB)

Additional details

Dates

Available
2026

Software

Repository URL
https://github.com/bosun-sh/papers/tree/main/intervention-penalty/goals-harness
Programming language
TypeScript

References

  • Shyam Agarwal, Hao He, and Bogdan Vasilescu. AI IDEs or autonomous agents? measuring the impact of coding agents on software development. arXiv preprint arXiv:2601.13597, 2026.
  • Analytics India Magazine. Human-in-the-loop is out, agent-in-the-loop is in. https://analytic sindiamag.com/ai-highlights/human-in-the-loop-is-out-agent-in-the-loop-is-in/, 2025. Agent-in-the-Loop (AITL) replacing HITL in leading Global Capability Centres; agents handle 90-99% of tasks. Accessed 2026-04-02.
  • Harry E. Chambers. My Way or the Highway: The Micromanagement Survival Guide. Berrett- Koehler Publishers, San Francisco, CA, 2004. ISBN 9781576752968.
  • CIO. Keeping humans in the AI loop. https://www.cio.com/article/4042910/keeping-human s-in-the-ai-loop.html, 2025. Alert fatigue causes humans to miss critical issues; advocates human-in-command over real-time review. Accessed 2026-04-02.
  • European Parliament and Council of the European Union. AI act: Regulation (eu) 2024/1689 of the european parliament and of the council. Technical report, Official Journal of the European Union, 2024. Article 14 mandates human oversight measures for high-risk AI systems. Entered into force August 2024.
  • K. J. Kevin Feng, David W. McDonald, and Amy X. Zhang. Levels of autonomy for AI agents. arXiv preprint arXiv:2506.12469, 2025.
  • Fortune/Eye on AI. AI tools outperform human professionals in law and advertising. https: //fortune.com/2025/12/09/ai-tools-outperform-human-professionals-law-adverti sing-ai-alone/, 2025. AI-only solutions outperformed human+AI centaur approach in legal research and ad design. Accessed 2026-04-02.
  • Xinyue Hao, Emrah Demir, and Daniel Eyers. Beyond human-in-the-loop: Sensemaking between AI and HI collaboration. Sustainable Futures, 10:100177, 2025. 28 interviews across 9 firms; reframes human-AI interaction as sociotechnical system requiring reflexive governance. DOI: S2666188825007166.
  • ImagineX Digital. Why your multi-agent AI system is probably making things worse. https: //www.imaginexdigital.com/insights/why-your-multi-agent-ai-system-is-proba bly-making-things-worse, 2026. Error amplification factor of 17.2 in multi-agent systems; coordination tax exceeds collaboration benefit. Accessed 2026-04-02.
  • An Luo, Jin Du, Xun Xian, Robert Specht, Fangqiao Tian, Ganghua Wang, Xuan Bi, Charles Fleming, Ashish Kundu, Jayanth Srinivasa, Mingyi Hong, Rui Zhang, Tianxi Li, Galin Jones, and Jie Ding. AgentDS technical report: Benchmarking the future of human-AI collaboration in domain-specific data science. arXiv preprint arXiv:2603.19005, 2025.
  • METR. Measuring the impact of early-2025 AI on experienced open-source developer productivity. https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/, 2025. 16 experienced OSS developers, 246 real issues; AI-assisted developers took 19% longer than AI-disallowed controls. Accessed 2026-04-02.
  • SeyyedAbdolHojjat MoghadasNian, Mona NaserPour Asiabari, and Ali HeidariYekta. AISA-L: Agentic AI strategy architecture for real-time KPI orchestration in sustainable, resilient airline logistics. SSRN preprint 6025895, 2025. Four-layer agentic architecture operationalizing 110 airline logistics KPIs; 22% forecast accuracy improvement.
  • Qodo. State of AI code quality 2025. https://www.qodo.ai/reports/state-of-ai-code-quali ty/, 2025. 65% of developers report AI misses relevant context during refactoring; 44% attribute degraded code quality to missing context. Accessed 2026-04-02.
  • Z. Rasheed, M. Waseem, M. Sami, K.-K. Kemell, A. Ahmad, A. Nguyen-Duc, K. Systä, and P. Abrahamsson. Autonomous agents in software development: A vision paper. In XP 2024 Workshops (Lecture Notes in Business Information Processing, Vol. 524). Springer, Cham, 2024. doi: 10.1007/978-3-031-72781-8_2. 12 LLM agents collaborating on full SDLC; significantly reduced development time.
  • ShiftMag. This CTO says 92.6% of developers use AI — but productivity is still only 10%. https://shiftmag.dev/this-cto-says-93-of-developers-use-ai-but-productivity-i s-still-10-8013/, 2026. 92.6% of developers use AI coding assistants; productivity plateau at ∼10% gain. Accessed 2026-04-02.
  • SiliconANGLE. Human-in-the-loop has hit the wall: Time for AI to oversee AI. https://sili conangle.com/2026/01/18/human-loop-hit-wall-time-ai-oversee-ai/, 2026. HITL is unscalable at modern AI decision velocity; advocates AI-governs-AI with human oversight at architectural level. Accessed 2026-04-02.
  • Stack Overflow. Stack overflow developer survey 2025: AI section. https://survey.stackoverfl ow.co/2025/ai, 2025. 46% actively distrust AI tool accuracy; only 3% highly trust AI output. Accessed 2026-04-02.
  • TDWI. The role of human-in-the-loop in AI-driven data management. https://tdwi.org/art icles/2025/09/03/adv-all-role-of-human-in-the-loop-in-ai-data-management.aspx, 2025. Risk-based calibration of human involvement proportional to decision consequence; HITL foundational in regulated sectors. Accessed 2026-04-02.
  • Unknown. KPIs for AI agents and generative AI: A rigorous framework for evaluation and accountability. ResearchGate preprint 392643274, 2024. Five-dimensional KPI framework (Model 25 Quality, System Performance, Business Impact, Human-AI Interaction, Ethical & Environmental); validated on GPT-4, DALL-E 3, Claude3.
  • Richard D. White. The micromanagement disease: Symptoms, diagnosis, and cure. Public Personnel Management, 39(1):71–76, 2010. doi: 10.1177/009102601003900105.
  • Yi Zheng, Chongyang Ma, Kanle Shi, and Haibin Huang. Agents meet OKR: An object and key results driven agent system with hierarchical self-collaboration and self-evaluation. arXiv preprint arXiv:2311.16542, 2023. 26