The Intent Gap: A Taxonomy of Real-User Failure Modes in Frontier AI Agents

Abstract
Public benchmarks for frontier AI agents (GAIA, OSWorld, AgentBench, GDPval,
BrowseComp, SWE-bench) are uniformly researcher-imagined: tasks are
designed by AI researchers, in clean declarative form, with ground-truth answers
fixed in advance. Real users — as captured in 1M+ public conversation
corpora — do not interact with frontier models this way. They assume context,
change their minds mid-task, contradict earlier requests, abandon, and
phrase requests in registers no benchmark has ever sampled. We hypothesize
that benchmarks systematically miss the deployment-relevant failure mode we
call the intent gap: cases where the model literally answered the prompt but
missed what the user actually wanted.
We mine WildChat-1M and LMSYS-Chat-1M using a three-stage frustrationsignal
filter (regex → embedding → LLM-as-judge) to surface a target of ~150–
300 high-quality intent-gap conversations. We replay each prompt across four
current frontier models (Claude, GPT, Gemini, and one open baseline) and
grade each replay on two axes: literal compliance and intent satisfaction. We
then triangulate the resulting taxonomy against ~25 published failure-mode taxonomies
from research institutes worldwide and against 5 documented production
failure incidents (lawsuits, regulatory filings, public post-mortems).
Headline preliminary findings (from a 50,000-conversation v2 pilot,
before cross-model replay or full hand-review):
1. 52.8% of English real-user conversations end after a single exchange.
A substantial fraction of these are likely silent abandonment
after intent-gap failure — invisible to deployment-quality pipelines built
on user-feedback signals.
2. Only 3.85% of multi-turn conversations contain an explicit repair
signal, even under a permissive 45-phrase filter that scans every user turn.
The intent gap exists, but users rarely verbalize it.
3. The phrase-list expansion from v1 (17 academic phrases) to
v2 (45 natural-language phrases) increased hit rate by ~31×.
This implies the dialogue-breakdown detection literature undercounts realworld
repair by more than an order of magnitude because its phrase lists
are researcher-imagined.
4. Among successfully-judged v2 candidates, 79% (15/19) were
confirmed as real intent-gap failures by an LLM judge. When the regex fires and the judge can run, the flagged candidate is overwhelmingly
a real failure.
5. The judge model refused to grade 62% (31/50) of real-user candidates,
most plausibly on content-moderation grounds. This is itself a
deployment-relevant finding: safety-filtered judge models systematically
miss failures in the most permissive parts of the user-prompt distribution.
A hand-reviewed and LLM-judge-filtered corpus of ~150–300 high-quality intentgap
conversations is released alongside the filter pipeline, the grading rubric, and
the triangulation matrix mapping our taxonomy onto ~25 prior published failure
taxonomies plus 5 documented production-failure incidents.