Toward an AI Personalization Index: A 157-Day Single-User Case Study
Description
This paper addresses a growing mismatch in the AI industry: frontier model benchmark scores are rapidly converging, yet most users still do not experience transformative productivity gains. While top model performance gaps have narrowed, large-scale workplace surveys show that AI usage is widespread but advanced utilization remains rare. The paper asks whether the missing variable is not raw model capability, but persistent accumulated personalization.
The paper proposes that experienced AI performance depends not only on model performance, but also on how well the system remembers, understands, and adapts to a specific user over time. It distinguishes behavior-tracking personalization, such as recommendation systems that optimize for clicks, from context-accumulating personalization, in which an AI system learns the user’s stable preferences, work context, communication style, and recurring goals.
The central contribution is a preliminary AI Personalization Index framework. The paper decomposes personalization into a baseline component available even in zero-memory sessions and an accumulated component that grows through long-term memory, conversation history, and preference learning. It proposes four candidate variables for future measurement: memory depth, conversation volume, context accuracy, and preference learning rate.
The empirical layer is an exploratory single-user case study. Over 157 days and roughly 1,000 hours of use on Claude, the author manually built and maintained a high-personalization workflow using 30 memory slots, persistent configuration artifacts, and explicit preference-learning rules. Because the underlying model was upgraded several times during the study period, the paper does not claim to isolate personalization effects from model improvement across the full timeline.
The strongest evidence comes from a same-day, same-model comparison between an established personalized project and a new zero-memory project on Claude Opus 4.6. Across 12 comparison days, the personalized project required far fewer context re-explanations, far fewer correction requests, and reached meaningful output substantially faster. The established project typically required 0–1 context re-explanations and 2–5 correction requests per session, while the new project required 5–7 re-explanations and 20–25 correction requests. Time to first meaningful output was about 5 minutes in the personalized project versus about 30 minutes in the new session.
The paper is explicit about its limits. This is an n=1 exploratory case study, not a controlled multi-user experiment. Several measurements are observational self-reports, task equivalence was not formally controlled, and general-user values are illustrative estimates rather than population data. The proposed Personalization Index is therefore a measurement scaffold, not a validated metric.
The conclusion is that persistent personalization may be a major hidden determinant of experienced AI performance. As benchmark performance gaps shrink, platforms may compete less on raw model intelligence alone and more on whether they can preserve user-specific context, reduce re-explanation burden, learn preferences safely, and make advanced utilization accessible to ordinary users.
The paper also outlines five platform-level design directions: automatic personalization database construction, automatic memory expansion, deeper retrieval-generation integration, automatic memory consolidation, and preliminary personalization-level measurement. Because personalization data can become a detailed model of the user’s identity, the paper proposes five data-sovereignty principles: user ownership, portability, right to erasure, consent granularity, and anti-stratification.
Keywords: AI personalization, personalization index, experienced performance, benchmark saturation, long-term memory, context accumulation, AI user experience, human-computer interaction, preference learning, memory systems, data sovereignty, AI productivity, advanced AI utilization.
Files
Lee_2026_AI_Personalization_Index_v3.pdf
Files
(162.0 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:2cb03e7c6600f492584e74e56295daa7
|
162.0 kB | Preview Download |
Additional details
References
- [1] Stanford University HAI, "The 2025 AI Index Report: Technical Performance," April 2025.
- [2] EY, "2025 Work Reimagined Survey," November 2025. (15,000 employees + 1,500 employers, 29 countries)
- [3] OpenAI, "The State of Enterprise AI: 2025 Report," December 2025.
- [4] VentureBeat, "OpenAI report reveals a 6× productivity gap between AI power users and everyone else," December 2025.
- [5] Adobe, "AI and Digital Trends 2026," February 2026.
- [6] Deloitte, "AI Expectations Report," 2025.
- [7] CX Today, "CX Trends 2025 Part 3: When AI Knows You Better Than You Know Yourself," December 2025.
- [8] Microsoft Advertising, "The future of AI personalization is inclusive," June 2025.
- [9] IEEE Spectrum, "The State of AI 2025: 12 Eye-Opening Graphs," May 2025.
- [10] Futurism, "Scientists Are Getting Seriously Worried That We've Already Hit Peak AI," August 2025.
- [11] AI2Work, "AI Progress Plateau in 2025: Economic and Strategic Implications," October 2025.
- [12] HoneHQ, "The AI Performance Gap: Why Investment Isn't Translating Into Impact," January 2026.
- [13] Anthropic, "Bringing memory to Claude," https://www.anthropic.com/news/memory, September 2025.
- [14] Anthropic, "Introducing agent threads," https://www.anthropic.com/news/agent-threads, March 2026.
- [15] Anthropic, "Claude Code: Memory management," https://docs.anthropic.com/en/docs/claude-code/memory, March 2026.
- [16] Anthropic, "Memory import," https://docs.anthropic.com/en/docs/memory-import, March 2026.
- [17] Anthropic, "Computer use," https://docs.anthropic.com/en/docs/computer-use, March 2026.
- [18] B. Gates, "Interview: The future of AI," Handelsblatt, January 22, 2023.
- [19] C. Packer, S. Wooders, K. Lin, V. Fang, S. G. Patil, I. Stoica, and J. E. Gonzalez, "MemGPT: Towards LLMs as Operating Systems," arXiv:2310.08560, 2024.
- [20] W. Zhong et al., "MemoryBank: Enhancing Large Language Models with Long-Term Memory," AAAI, 2024.
- [21] H. Li, C. Yang, A. Zhang, Y. Deng, X. Wang, and T.-S. Chua, "Hello Again! LLM-powered Personalized Agent for Long-term Dialogue," NAACL 2025, pp. 5259-5276.
- [22] Z. Pan, Q. Wu, H. Jiang et al., "SeCom: On Memory Construction and Retrieval for Personalized Conversational Agents," ICLR, 2025.
- [23] X. Li et al., "A Survey of Personalization: From RAG to Agent," arXiv:2504.10147, 2025.
- [24] Z. Tan et al., "In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents," ACL 2025.
- [25] J. S. Park, J. C. O'Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein, "Generative Agents: Interactive Simulacra of Human Behavior," UIST, 2023.
- [26] Y. Shin, "Toward Human-Centered Artificial Intelligence for Users' Digital Well-Being," JMIR Human Factors, vol. 12, e69533, 2025.
- [27] H. R. Kirk, B. Vidgen, P. Röttger, and S. Hale, "Personalisation within Bounds: A Risk Taxonomy and Policy Framework for the Alignment of Large Language Models with Personal Values," EMNLP 2024.
- [28] A. Salemi, S. Mysore, M. Bendersky, and H. Zamani, "LaMP: When Large Language Models Meet Personalization," arXiv:2304.11406, 2024.
- [29] J. Zhang, K. Chen, Y. Liu et al., "A Survey on Personalized Large Language Models," arXiv:2502.15836, 2025.