Composition Hallucination in Retrieval-Augmented Generation: A Failure Mode and Benchmark Protocol

Retrieval-Augmented Generation (RAG) is commonly motivated by the idea that language models answer more faithfully when relevant evidence is retrieved and placed in context. This assumption is useful but incomplete. In policy, legal, clinical, educational, and operational documents, correct answers often depend not only on relevant fragments but on relations among them: exceptions, overrides, scopes, preconditions, precedence, temporal dependencies, and procedural order. This paper defines composition hallucination: a failure mode in which a model produces an answer incompatible with information present in its context, even though the necessary fragments are available and locally interpretable, because the model fails to compose those fragments according to their implicit relations. We distinguish composition hallucination from parametric hallucination, retrieval failure, and long-context or positional context-use failures. We then propose a benchmark protocol for isolating the phenomenon without reporting new model results. The protocol uses paired cases in which the same informational content is presented under different relational explicitness conditions: raw prose, relation-annotated prose, and structurally compiled representations. Each case includes sufficiency checks, local-legibility probes, counterfactual controls, position controls, and retrieval controls so that failures can be attributed to composition rather than missing evidence, unreadable fragments, or long-context decay. The purpose is methodological: to make a frequent but under-specified failure mode measurable, falsifiable, and separable from adjacent RAG failure modes.