A Governance Methodology Layer for AI-Assisted Software Development: Defect Taxonomy, Controlled Ablation, and Process-Over-Capability Evidence
Authors/Creators
Description
Autonomous coding agents produce output that passes syntactic checks — compilation, type safety, CI — at high velocity. Yet syntactic correctness does not imply semantic correctness: design boundaries, security invariants, and maintainability contracts remain structurally invisible to automated pipelines. This paper makes four contributions toward closing this gap. First, we present a defect-class taxonomy grounded in five AI agent permission and governance modules, distinguishing defects structurally detectable by static analysis from those requiring semantic review. Second, we describe a runtime-decoupled governance gate — a file-based protocol that reads generator output and emits a structured verdict without API coupling, making it portable across code-generation tools. Third, we formalize methodology-as-code: expressing a verification protocol as a version-controlled, executable, cross-platform artifact with a two-layer architecture separating portable methodology from host-specific automation. Fourth, we report a controlled ablation experiment (E-ablation, N=5 artifacts, 8-item independent ground truth) comparing harness-structured review against a token-matched unstructured review prompt on the same artifacts. The structured condition records 62% lenient ground-truth recall against 50% for the unstructured condition, with 25% strict recall against 0%, and applies more consistent severity grades where the unstructured condition inflates severity. Both conditions miss document-quality defects identified by a human QA reviewer, indicating a complementarity between structured AI review and human process inspection. Within the bounds of this sample, the results are consistent with the thesis that review-process design, rather than reviewer-model capability, is the dominant factor in critical-defect coverage for AI-assisted software development.
Companion to "The Forge Harness: A Structured Methodology for Sustainable AI-Assisted Software Development" (Zenodo 10.5281/zenodo.20397566).
Files
forge-harness_cs-se_governance-methodology_2026.pdf
Files
(729.8 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:0f7ebf410880c435caed2c52cc616c70
|
729.8 kB | Preview Download |
Additional details
Related works
- Cites
- arXiv:2603.25723 (arXiv)
- Continues
- 10.5281/zenodo.20397566 (DOI)
- Is supplemented by
- https://github.com/chrono-meta/forge-harness (URL)
Software
- Repository URL
- https://github.com/chrono-meta/forge-harness
- Development Status
- Active