Manifold: Eliminating False Positives in Multi-Agent LLM Systems Through Specification-Driven Orchestration
Description
Multi-agent LLM systems promise autonomous task execution but suffer from two
critical reliability failures: false positives (systems report success on tasks that fail
strict validation) and retry inflation (traditional retry logic degrades performance while
increasing costs). We demonstrate that standard validation approaches produce a 66%
false positive rate on structured extraction tasks, with naive prompting achieving 100%
reported success but only 34% true success under universal validation criteria. We present
Manifold, a specification-driven orchestration architecture that treats specifications
as verifiable contracts between agents. Manifold combines the Specification Pattern
from object-oriented design with fingerprint-based loop detection to prevent infinite
retry cycles while ensuring output correctness. Across 600 controlled trials spanning
four domains (adversarial image generation, structured data extraction, and multi-step
synthesis), Manifold achieved 94% true success rate versus 34% for naive prompting
(p < 0.001, Cohen’s h = 1.40) with zero false positives compared to naive’s 66%
false positive rate. On structured extraction tasks, Manifold produced 99.1% field-level
accuracy while eliminating all false positives. Smart control (retry logic) degraded to
58–98% success rates while inflating costs by 1.5–3.5× across all experiments. Our
results demonstrate that specification-driven validation enables trustworthy autonomous
operation by providing verifiable correctness guarantees, with implications for production
LLM deployment at enterprise scale.
Files
Manifold__Eliminating_False_Positives_in_Multi_Agent_LLM_Systems_Through_Specification_Driven_Orchestration (2).pdf
Files
(577.8 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:b012cc7341b4ac6fa2c48dc7b2e28643
|
577.8 kB | Preview Download |
Additional details
Software
- Repository URL
- https://github.com/fabs133/manifold
- Programming language
- Python
- Development Status
- Active