Published November 12, 2026 | Version 1.0.0

Headroom Decision Procedure Examples: dataset for "Headroom Before Protocol Effects: A Decision Procedure for Human-Agent Evaluation"

  • 1. Metacognition Institute
  • 2. Clemson University

Description

Synthetic worked example for the headroom decision procedure: worked-headroom, planning-headroom, severity-scaled and guard-sensitivity result tables, and the scripts that generate them.

Dataset accompanying the paper Headroom Before Protocol Effects: A Decision Procedure for Human-Agent Evaluation, Human-AI Coevolution Workshop (HAIC) at NeurIPS 2026 (https://openreview.net/forum?id=odOgsQYleZ).

Paper abstract. A team protocol has no interpretable benefit without a matched no-protocol baseline and decision-relevant headroom. We present a reporting method that tests admissibility, identifies the marginal contrast, evaluates opportunity, and maps paired uncertainty to an operational action. It distinguishes exact headroom on a fixed battery from uncertain population headroom used in planning. A published meta-analysis of human-AI experiments illustrates why the estimand matters: the same evidence supports augmentation relative to humans alone but contradicts synergy relative to the better constituent. Neither comparison identifies a coordination protocol without a matched no-protocol team. An executable synthetic application further shows that a promising point estimate can remain operationally unresolved once paired uncertainty is considered, and that near-perfect pilot performance does not by itself make a modest benefit impossible. Several inherited multi-model summaries fail the method's artifact gates, so they do not validate it. This is a bounded position and reporting contribution rather than an adoption-ready standard. Human-facing claims still require direct human outcomes, and we introduce no new human experiment.

Known limitations.

  • All tables are synthetic illustrations; the five earlier pilots audited in the paper have no retained raw data.

License: Creative Commons Attribution 4.0 International (https://creativecommons.org/licenses/by/4.0/). Source code files are released under the MIT License.

Files

croissant.json

Files (44.8 kB)

Name Size
md5:f66e8cfcbe26c7be32b917f83bcf29c3
1.2 kB Download
md5:1e00c423c77ead6383bcc5093c122091
20.4 kB Preview Download
md5:2abe9b6f418c6ee6036a095e8e8352ac
16.1 kB Preview Download
md5:dce724e3e39e198fe91dc9c2046a1c5d
1.6 kB Download
md5:ad05064be0c2e4e93dcf60d256139738
5.5 kB Preview Download

Additional details

Related works