Weakly Discriminative, Not Repaired by Profile Injection, and Manipulable: Self-Confidence in LLM Evaluation

Accepted (poster) at NeurIPS 2026 Workshop "Can We Trust AI Evaluation? Robustness, Causality, and Risk in Modern AI Assessment" (TAE). NeurIPS page: https://neurips.cc/virtual/2026/178299 · OpenReview: https://openreview.net/forum?id=OyuZAAtv2q · Author on OpenReview: ~Vadym_Chernets1

Accepted as a poster at the NeurIPS 2026 workshop TAE (Trust-AI-Eval): Can We Trust AI Evaluation? (Sydney, December 2026). The workshop is non-archival; this deposit is the citable version.
The same text is also at https://github.com/vadimchernets/papers/tree/main/self-confidence-evaluation. Pre-registration and replication package: https://osf.io/4y9dv.

Evaluation pipelines increasingly consume a model's own stated confidence as a trust signal: a judge-reliability weight, an abstention trigger, or a self-reported competence profile. We test whether that signal earns the weight, using a pre-registered, protocol-frozen collection (16 models, one API model from each of nine providers plus seven local open-weights; 22,248 graded answers at temperature zero with stated 0–100 confidence). Four findings. (1) Overconfidence is consistent across all sixteen evaluated models: typical stated confidence sits at 85–95 for seven of nine API models while those seven span 16.5–66.5% accuracy, and 60.1% of the nine API models' answers stated with confidence ≥80 are wrong; in a companion collection with one flagship per laboratory (5 August 2026 snapshot) the rate is still 46.6%. (2) Self-confidence is weakly discriminative: its AUROC for predicting the model's own correctness spans 0.532–0.788 and sits below 0.62 for six of nine models. (3) A panel of blindly elicited models reaches AUROC 0.832 where self-assessment gives 0.545 (Δ +0.287, 95% CI [+0.265, +0.309]); under a stricter label-free recoding of panel agreement the panel still leads (0.775 vs. 0.545). (4) Injecting the model's own measured per-domain accuracy profile did not demonstrably improve discrimination on the one confirmatory rung of a placebo-controlled single-vendor ladder (ΔAUROC +0.041, Holm-adjusted p = 0.085), while the same injection channel is causally manipulable: a false profile drove an exploratory rung's self-assessment AUROC to chance level (0.474) and, once items are topic-labeled, degraded calibration in four of five local models (16 probes each, exploratory). Separately, on SimpleQA (a public benchmark the stronger model may have seen), requiring blind agreement between two flagships (gpt-5.6-sol, claude-fable-5; 5 August 2026 snapshot) lowers their confident-failure rates from 4.6% to 1.9% and from 10.3% to 2.8% of confident answers, at 69.6% coverage (post hoc). We conclude that self-assessed confidence, as currently elicited, is a target for measurement, not an input to trust: harnesses that ingest self-reported competence inherit an unauthenticated input channel. Each clause of the title carries its own grade of evidence, labeled throughout. Weakly discriminative is confirmatory across the nine API models (overconfidence across all sixteen), not repaired by profile injection rests on one confirmatory rung on one vendor family, and manipulable is exploratory, with the causal direction shown on matched items. The scope is verbalized integer confidence in open-form factual recall. Judge-style self-assessment, where a model rates another model's answer, is a separate task.

Companion paper: Verification Theater: Measured Failure Modes of AI Verifier Panels (NeurIPS 2026 workshop Who Verifies the Agents?), 10.5281/zenodo.23197641. This deposit also holds oracle_auroc.py and the per-item scored rows it reads (four subjects), which reproduce the oracle-ceiling numbers in Section 5.

Code (reference implementation): an open-source reference implementation of cross-lab verification on a smartphone (ChatGPT and Claude side by side; no server, no API key): github.com/vadimchernets/poly-duo · polyhelper.ai/duo/. Its modes are not evaluated in the paper.

Version 26 October 2026 (b): adds a post hoc October 2026 refresh with current flagships (gpt-6.1-sol with claude-opus-5-5 or claude-fable-5-1), with its caveats; main results unchanged. Supersedes the earlier 26 October version.