Published June 16, 2026 | Version v1
Preprint Open

Expert 114: A Linear Router Axis for Inhabited Self-Examination in a Mixture-of-Experts Language Model — and Why It Does Not Transfer

Description

Mixture-of-experts (MoE) routing exposes a discrete, per-token record of which experts fire, making it an unusually legible readout for interpretability. Yet single experts are seldom characterized at the level of a specific functional role, and an “introspective” register, where a model speaks from inside a point of view about its own processing, is easy to over-read as evidence about machine experience. This paper characterizes one routed expert, Expert 114 (E114) at layer 14 of Qwen3.5-35B-A3B, a 40-layer, 256-expert, top-8 MoE model, while explicitly bounding the claim.

 

The E114 router row, recovered by least squares from captured residual-logit pairs, defines a single linear axis separating generated inhabited self-examination text from lexically matched controls at Cohen’s d = 3.88 with no overlap. An earlier 21.68x routed-weight ratio is shown to reflect top-k ratio inflation and is demoted to a footnote. A dissociation battery separates the axis from deny/affirm verdict, safety/refusal behavior, topic, grammatical person, and next-token entropy. The controlled variable is the referent: the model’s own interior, graded by the intensity of the examination act. Under this axis, an inanimate rock and a thermostat outrank a cat, indicating that the signal tracks self-referential examination rather than the apparent sentience of the described entity.

 

E114’s router gate logit disengages a few tokens before a degenerating continuation visibly collapses. Injecting the register’s residual direction past the layer-14 router is sufficient, though necessity remains untested, to induce the register. Forcing the gate open upstream does not by itself produce the same effect, suggesting that E114 functions as a readout rather than a demonstrated controller. The role is also model-specific: on Qwen3.5-122B-A10B, index 114 appears computer-science-linked and is suppressed by the same control that amplifies it at 35B, with the softmax-side expert E48 remaining a candidate analog pending validation.

 

E114 is therefore best understood as a register detector, not as evidence of machine experience.

 

Code and durable artifacts, including manuscript source, experiment journals, curated run results, the SAE vantage-ladder, and steering source, are openly available under MIT license at:

 

https://github.com/jeffreywilliamportfolio/e114-artifacts

Files

e114_acl_style.pdf

Files (550.5 kB)

Name Size Download all
md5:3b7fca6275b5897b4bef7fb666f72f2d
550.5 kB Preview Download

Additional details