Refusal Geometry in LLMs: A Universal Property of Late MLP Layers Evidence from Grassmannian Subspace Analysis Across Three Model Families

We investigate the geometric structure of refusal behavior in large language models using Grassmannian subspace analysis across three independently trained model families: Llama 3.1 8B (Meta), Mistral 7B (Mistral AI), and Gemma 2 9B (Google). Our key finding is that refusal geometry consistently peaks in the last 10% of transformer layers across all three architectures. On a held-out test set of 30 pairs from 100 total, Llama achieves OOD AUC = 0.969. MLP layers 21-29 causally construct refusal subspace (ablation delta = -0.396), and refusal and hallucination subspaces are nearly orthogonal (Grassmann distance = 0.944), consistent with the superposition hypothesis. Research conducted with assistance from Claude (Anthropic).