Cross-Model Evaluation of Symbolic Persona Anchors and Natural-Language Reminders for Long-Context Persona Preservation
Description
Abstract
Large Language Models (LLMs) configured as specialized agents rely heavily on extensive system prompts to enforce behavioral, lexical, and stylistic personas. However, during extended multi-turn interactions, these models experience Persona Drift due to attention dilution across widening context horizons. While brute-force re-prompting introduces severe token overhead, this paper evaluates Symbolic Pseudonym Anchoring (SPA)—a context engineering framework that maps dense behavioral schemas to compact, model-safe symbolic identifier tokens—as a low-overhead intervention. Rather than claiming universal efficacy, we present a rigorous cross-model evaluation of SPA against natural-language reminders and structural baselines across ten distinct open-weight model architectures using a 7-arm ablation matrix. Empirical results show that SPA’s effectiveness is highly model-dependent: while Continuous SPA improved aggregate Psem relative to the baseline for specific architectures like Qwen2.5:3B and Gemma2:9B, it introduced reduced persona-preservation performance in others, such as the Llama-3 and Mistral variants. In cross-model aggregate performance, a standard Generic Reminder achieved the highest overall persona preservation score (0.686), whereas Continuous SPA and Drift-Triggered SPA achieved 0.673 and 0.677 respectively. These findings suggest that symbolic token anchoring does not universally outperform natural-language constraints but serves as a highly conditional, model-specific intervention.
Files
Cross_Model_Evaluation_of_Symbolic_Persona_Anchors_and_Natural_Language_Reminders_for_Long_Context_Persona_Preservation.pdf
Additional details
Software
- Programming language
- Python
References
- H. K. Choi and Y. Li, "PICLe: Eliciting Diverse Behaviors from Large Language Models with Persona In-Context Learning," arXiv preprint arXiv:2405.02501, 2024.
- K. Li et al., "Measuring and Controlling Instruction (In)Stability in Language Model Dialogs," arXiv preprint arXiv:2402.10962, 2024.
- N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang, "Lost in the middle: How language models use long contexts," Transactions of the Association for Computational Linguistics, vol. 12, pp. 157–173, 2024.
- [Learning to Compress Prompt in Natural Language Formats](https://aclanthology.org/2024.naacl-long.429/) (Chuang et al., NAACL 2024).