In-Context Availability Reorganizes Expert Routing: A Measurement Confound for Multi-Turn Evaluations of Mixture-of-Experts Language Models

In-Context Availability Reorganizes Expert Routing: A Measurement Confound for Multi-Turn Evaluations of Mixture-of-Experts Language Models. Preprint, version 1.1 (August 2026). Not peer reviewed.

In a sparse mixture-of-experts (MoE) language model the router selects a small subset of experts for every token, and that selection is increasingly read as an internal-state measurement: to characterize specialization, to audit safety-relevant behavior, and to track reasoning state. We show that one property of the preceding context, whether the text being processed is already present verbatim in an earlier turn (in-context availability), reorganizes routing at scale at fixed token identity and fixed final prompt. In Qwen3.5-35B-A3B-Base, teacher-forced response pairs differing only in the prior turn shifted layer-14 selection of pre-specified register-specific expert sets by -0.175 (95% prompt-cluster bootstrap confidence interval [-0.184,-0.166]) and raised a pre-specified cross-register set by +0.058 ([0.055,0.061]), under expert sets, estimands, and decision rules fixed before capture. A 2×2 factorial attributes the shift to literal availability of the target, with prompt repetition contributing under one tenth, and the shift survives matching on predictive entropy and target surprisal. Attention retrieval of the prior copy carries about half of it: under availability roughly half of each target token's attention mass lands on the prior copy, and masking that attention moves layer-14 routing more than half of the way back to its no-availability configuration (12 of 12 pairs). The set structure replicates on instruction-tuned Qwen3.6-35B-A3B under seven expert-level predictions fixed before capture (7 of 7 confirmed), yet excluding or forcing the confirmed experts at layer 14 during generation leaves output token-identical in every prompt tested. The signature is therefore a reliable correlate of the availability condition that carries no measurable behavioral load at its layer. The consequence is methodological: internal-state measurements taken on multi-turn probes that place later target text in earlier context partly measure availability in addition to the capability under test, and replicated observational routing signatures do not by themselves license causal claims about the experts involved.

Changes in this version. Version 1.1 (August 2026): single-thesis revision of 1.0.2; one claim carried abstract to conclusion; no reported value changed.