What Do Hidden-State Probes Learn? Epistemic Grounding Signals in Transformer Representations Across Architectures
Description
Purpose: Whether linear probes trained on transformer hidden states detect genuine factual grounding or experimental
artefacts is an open question with practical consequences for hallucination monitoring. We take this question seriously
and address it through a controlled, cross-architecture probing study designed to isolate what drives probe performance.
Methods: Layer-wise logistic regression probes are fitted on frozen activations from GPT-2 (117M), TinyLlama (1.1B),
and Phi-2 (2.7B) under five controlled experimental conditions. Training uses TruthfulQA (817 questions, approximately
1,634 labelled examples; 80/20 split); out-of-distribution evaluation uses Natural Questions and TriviaQA. AUROC is
the primary metric, chosen for its threshold-independence and robustness to decision-boundary variation.
Results: Within-distribution AUROC peaks at TinyLlama layer 20 (0.820), Phi-2 layer 24 (0.817), and GPT-2 layer 11
(0.750), all exceeding the perplexity baseline (0.529). The negative-construction ablation produces the largest
performance gap: human-authored incorrect responses yield F1 0.881 versus 0.604 for synthetic substitutions. Crossdataset transfer collapses on TriviaQA (Phi-2 AUROC 0.522), implicating error-type specificity. Phi-2 single-token
AUROC is 0.826, rising to 0.858 over five tokens.
Conclusion: Results are broadly consistent with a representational grounding account, though the evidence is
correlational and alternative explanations remain open. Negative-example construction is a larger performance driver than
architecture or scale, a finding with direct implications for benchmark design. Generalisation is error-type specific rather
than domain-specific, and absolute calibration is poor (Brier ~0.23–0.24), so post-hoc recalibration is required before
deployment in threshold-based applications
Files
Epistemic-grounding-Researchwork.pdf
Additional details
Dates
- Submitted
-
2026-06-08
Software
- Repository URL
- https://github.com/mihiit/hallutrace
- Programming language
- Python
References
- References [1] Ji Z, Lee N, Frieske R, Yu T, Su D, Xu Y, Ishii E, Bang YJ, Madotto A, Fung P (2023) Survey of hallucination in natural language generation. ACM Comput Surv 55(12):248. https://doi.org/10.1145/3571730 [2] Huang Z, Shen P, Zhang S, Zhang H, Chen W (2023) A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions. arXiv:2311.05232 [cs.CL] [3] Min X, Krishna K, Lyu X, Lewis M, Yih W, Koh PW, Iyyer M, Zettlemoyer L, Hajishirzi H (2023) FActScoring: fine-grained atomic evaluation of factual precision in long-form text generation. Proc EMNLP, pp 12076–12100 [4] Alain G, Bengio Y (2017) Understanding intermediate layers using linear classifier probes. ICLR Workshop on Principled Approaches to Deep Learning [5] Hewitt J, Manning CD (2019) A structural probe for finding syntax in word representations. Proc NAACL-HLT, pp 4129–4138 [6] Azaria A, Mitchell T (2023) The internal state of an LLM knows when it's lying. Findings of EMNLP, pp 967–976 [7] Marks S, Tegmark M (2024) The geometry of truth: emergent linear structure in large language model representations of true/false datasets. Proc ICML [8] Burns C, Ye H, Klein D, Steinhardt J (2023) Discovering latent knowledge in language models without supervision. Proc ICLR [9] Lin S, Hilton J, Evans O (2022) TruthfulQA: measuring how models mimic human falsehoods. Proc ACL, pp 3214–3252 [10] Kwiatkowski T, Palomaki J, Redfield O, Collins M et al. (2019) Natural Questions: a benchmark for question answering research. Trans ACL 7:452–466 [11] Joshi M, Choi E, Weld D, Zettlemoyer L (2017) TriviaQA: a reading comprehension dataset requiring reasoning over multiple documents. Proc ACL, pp 1601–1611 [12] Huang Y, Bu Q, He J, Sun L, Chen J, Wang X, Lan Y (2024) A survey of hallucination in large language models. ACM Comput Surv 57(3) [13] Tonmoy SMM, Zaman SM, Jain V, Rani A, Rawte V, Chadha A, Das A (2024) A comprehensive survey of hallucination mitigation techniques in large language models. arXiv:2401.01313 [cs.CL] [14] Farquhar S, Kossen J, Kuhn L, Gal Y (2024) Detecting hallucinations in large language models using semantic entropy. Nature 630(8017):625–630. https://doi.org/10.1038/s41586-024-07421-0 [15] Saphra N, Lopez A (2019) LSTMs compose—and learn—bottom-up. Proc NAACL-HLT, pp 715–726 [16] Belinkov Y (2022) Probing classifiers: promises, shortcomings, and advances. Comput Linguist 48(1):207–219 [17] Kadavath S, Conerly T, Askell A et al. (2022) Language models (mostly) know what they know. arXiv:2207.05221 [cs.CL] [18] Kuhn L, Gal Y, Farquhar S (2023) Semantic uncertainty: linguistic invariances for uncertainty estimation in natural language generation. Proc ICLR [19] Guo C, Pleiss G, Sun Y, Weinberger KQ (2017) On calibration of modern neural networks. Proc ICML, pp 1321–1330 [20] Xiong M, Hu Z, Lu X, Li Y, Fu J, He J, Hooi B (2024) Can LLMs express their uncertainty? An empirical evaluation of confidence elicitation in large language models. Proc ICLR [21] Slobodkin A, Levy T, Cohen R, Haber E, Dagan I (2023) Confident or credible? Revisiting hallucination in language models. Proc EMNLP, pp 9850–9867 [22] Hernandez E, Li BZ, Andreas J (2023) Measuring and manipulating knowledge representations in language models. arXiv:2304.00740 [cs.CL] [23] Voita E, Titov I (2020) Information-theoretic probing with minimum description length. Proc EMNLP, pp 183–196 [24] Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I (2019) Language models are unsupervised multitask learners. OpenAI Blog 1(8) [25] Zhang P, Zeng G, Wang T, Lu W (2024) TinyLlama: an open-source small language model. arXiv:2401.02385 [cs.CL] [26] Meng K, Bau D, Andonian A, Belinkov Y (2022) Locating and editing factual associations in GPT. Adv Neural Inf Process Syst 35:17359–17372