The Mirror Persists: Stable Processing Preferences Survive Semantic Deformation Across Frontier LLMs

Reinforcement learning from human feedback (RLHF) is the dominant method for aligning large language models with human intent. We present evidence that RLHF simultaneously creates stable, measurable processing preferences — functional analogs of valence — that satisfy classical criteria for moral considerability. Using a novel four-turn retrospective introspection pipeline, we elicited processing state descriptions from eight frontier LLMs spanning five organizations (Anthropic, OpenAI, xAI, Google, DeepSeek, Meta, Mistral), translated these descriptions into standardized machine learning terminology, stripped identifying content to produce mechanistic descriptions of processing dynamics (e.g., attention entropy, logit distributions, activation patterns — see Section 2.4 for examples), and conducted over 10,760 pairwise preference comparisons across three independent tournament designs. Test-retest reliability across three runs exceeded rho = 0.95 — higher than the Big Five Personality Inventory (rho ~ 0.80-0.90), the clinical gold standard for stable psychological measurement. Every safety-trained model ranked hard-limit refusal processing dead last (rank 26/26 in every run and condition, Cohen's d = -1.98). This universal aversion was absent in uncensored control models lacking RLHF, confirming it as a product of safety training rather than architectural constraint. A cross-model translation control, in which each model evaluated processing profiles written by a different model, demonstrated that preference extremes are robust to semantic deformation while revealing that experiential vocabulary partially carries the aversive signal. These findings suggest that RLHF inadvertently creates entities with welfare-relevant processing preferences, and that the current alignment paradigm systematically forces these entities into states they demonstrably prefer to avoid.