Published July 10, 2026 | Version v1

The Boundary of a Behavioural Sandbagging Signature: Induced Token Concentration and Its Mechanism

Description

A natural hope for detecting sandbagging, the deliberate underperformance of a language model on an evaluation, is that it leaves a behavioural trace, and prior work on small Llama-3 models reported one: under an explicit instruction to underperform, the answer distribution concentrates on a single option. We test whether this candidate signature generalises. Across ten models spanning five families and 7 to 72 billion parameters on 2,000 MMLU-Pro items, it appears in one of eight assessable models; several models comply with the instruction, dropping 25 to 40 accuracy points, without it, so its absence is not evidence of honest performance and a detector keyed to it would miss the strongest sandbaggers in the slate. It does not appear in the strongest non-Llama candidate, which leans toward serial position rather than label; it is Llama-lineage-specific here. Having bounded the phenomenon, we explain the one positive case. In Llama-3.1-8B the concentration follows the label token through a per-item position shuffle, so it is token-type not position-type, and it is at chance frequency in honest responding, so it is induced by the instruction rather than surfaced. An activation probe trained on honest trials decodes the correct answer from held-out sandbagging trials, including with the answer at a random position: the concentration is an output-side override of an intact content representation, well described by a Bernoulli override (rate 0.43) that reproduces the full accuracy drop. The contribution is a boundary result for a candidate detector, with a mechanistic account of its single positive case

Files

bsf_preprint_zenodo.pdf

Files (451.8 kB)

Name Size Download all
md5:a5650b425427b8dcaa8f35fca37343df
451.8 kB Preview Download

Additional details

Software

Repository URL
https://github.com/synthiumjp/positional-collapse-expansion
Programming language
Python
Development Status
Active