The Appearance of Meaning: Context-Dependence and Semantic Competence in Transformer Architectures

Preprint. Under review at the Journal of Logic, Language and Information.

Large language models routinely elicit interpretations of meaning and context-sensitivity from competent
speakers. This paper does not argue that such models possess meaning proper. Instead, it presents a protocol-
and-constraint framework that isolates an empirical explanandum—appearance of meaning (AoM)—understood
as a competence profile exhibited under controlled contextual variation: (i) context-sensitive disambiguation,
(ii) selective sensitivity to meaning-altering edits over meaning-preserving shams, and (iii) discourse-level
constraint tracking beyond length-matched controls.
The central question is mechanistic and philosophically diagnostic: in transformer language models, what
internal variables causally control these meaning-like preference patterns when weights are held fixed? We
test a Context-Primacy Thesis (CPT): that donor-directed changes in AoM-relevant preference margins can
be induced by intervening on contextualized token-in-context states at characteristic depths, beyond sham
baselines. Across GPT-2 and Qwen2.5 checkpoints, activation patching yields structured, depth-localized
donor-directed effects with near-zero sham controls.
A fixed-depth target-specificity stress test further indicates that, under the canonical nearby matched-span
control used in Table 3, point estimates are positive across the non-GPT-2 models while GPT-2 remains a
negative case, with the clearest support in the Qwen models and more heterogeneous support among the
Llama models. We therefore interpret SDH as a local matched-span concentration result rather than as full
semantic specificity. The result is disciplined mechanistic constraint-setting for philosophical interpretations:
an empirically anchored account of context-dependence in transformer computation that informs, without
settling, disputes about semantic competence, reference, and normativity.