Published October 29, 2026 | Version v1

Cost per Verified Task: Replay-Based Evaluation of Back-Office Agents

Authors/Creators

  • 1. Independent researcher

Description

Cost per attempted task can conceal expensive failures. We study cost per verified success on 30 synthetic back-office tasks, using deterministic replay of file and customer-record operations. A privileged oracle completes every task (95% Wilson interval: 88.6-100.0%); each of six trivial policies completes none (0.0-11.4%). Ninety invalid-trace probes fail verification. A simulated population sweep compares direct execution with a redundant read-only control under a fixed artificial token tariff. The control preserves success and doubles assumed cost per success. These results validate accounting and replay behavior, not learned-agent ability or provider prices. No hosted model or human study is included.

Files

cost-per-task-paper.pdf

Files (87.2 kB)

Name Size
md5:e809c2e7b5eceac1682d85bb682ac2db
68.0 kB Preview Download
md5:47e711af549504616fa03eb883ae247c
19.2 kB Download

Additional details

Related works