Published August 26, 2025 | Version v1
Preprint Open

Do AI Agents Need Mentors? Evaluating Chain-Pattern Interrupt (CPI) for Oversight and Reliability

Authors/Creators

  • 1. MURST Initiative

Description

AI agents tackling complex, long-horizon tasks can get trapped in reasoning lock-in (RLI), where small misconceptions early in the task cascade into errors, wasted tokens and destructive outcomes. We introduce Chain-Pattern Interrupt (CPI), an external mentor mechanism that 'pauses' the agent at uncertainty or hazard points and elicits a mentor-like re-evaluation before continuing. We evaluate on two adversarial benchmarks: a debugging scenario and a priority-conflict scenario. With CPI, agents consistently deliver the requested outputs, avoid misleading suggestions, and roughly double success while reducing harmful actions by half. Our evaluation harness, logs, audit trails and replication instructions are released, enabling full reproducibility. Across 153 runs, success increased from 27% to 54% and harmful actions fell substantially; pooled OR 1.98 (CMH = 0.099). We show confidence intervals and full tests in Appendix B.

Files

Do_AI_Agents_Need_Mentors_Evaluating_CPI.pdf

Files (814.0 kB)

Name Size Download all
md5:aba768edfe1fdce58338ecd0e82036e4
814.0 kB Preview Download

Additional details

Dates

Available
2025-08-25

Software

Repository URL
https://github.com/PV-Bhat/cpi
Programming language
Python
Development Status
Active