The Agent-Readable Web: llms.txt, Companion Files, and Credential Verification in the ARO Framework
Description
This essay extends the Agentic Recommendation Optimization (ARO) framework to the emerging llms.txt standard and static markdown companion files. It proposes that curated markdown files, deployed alongside existing websites, can make licensed professionals structurally verifiable for autonomous AI agents that recommend service providers on behalf of users. The paper examines the token economics of HTML versus markdown delivery, documents adoption of the llms.txt standard across 2.2 million websites, distinguishes the approach from traditional search engine optimization, engages the strongest published counterarguments, and presents a falsifiable experimental design for testing whether markdown companion files measurably influence agent recommendation behavior. The application to credential verification for licensed local professionals is the novel contribution.
Notes (English)
The web is splitting into two parallel layers: one built for human visitors and one built for autonomous AI agents. This essay argues that static markdown companion files, deployed via the llms.txt standard proposed by Jeremy Howard, offer licensed professionals a low-cost path to verifiability in the emerging agent web. It introduces a three-layer site architecture (HTML for humans, prerendered HTML for search crawlers, static markdown for AI agents) and positions credential verification as the novel application of a standard that has so far been adopted almost exclusively by technology companies for developer documentation. The experimental design section proposes a controlled test comparing agent recommendation behavior on identical providers with and without markdown companion files. This is the second paper in the ARO series, extending the framework introduced in "Agentic Recommendation Optimization: A Blueprint for the Post-Search Economy" (Holmes, 2026).
Other (English)
This research is part of an ongoing project to define the standards for Agentic Recommendation Optimization (ARO). For further context, business applications, and professional verification, please refer to the following resources:
- Official Research Home: https://appraisermarketinggroup.com/Agentic-Recommendation-Optimization-Essay
- Substack (Deep Dives & Updates): https://georgechipholmes.substack.com/
- Author Personal Website: https://georgechipholmes.com
- Google Scholar Profile: https://scholar.google.com/citations?user=jXJTzyMAAAAJ
- ORCID Academic Identity: https://orcid.org/0009-0003-3956-0662
- Professional Identity (LinkedIn): https://www.linkedin.com/in/george-chip-holmes-67928520/
- First Paper in Series (Zenodo DOI): https://doi.org/10.5281/zenodo.18727694
Files
The Agent-Readable Web-llms.txt, Companion Files, and Credential Verification in the ARO Framework.pdf
Files
(28.0 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:35ca4636f3aec1166529bfe12dede6bf
|
28.0 kB | Preview Download |
Additional details
Related works
- Is supplement to
- Preprint: 10.5281/zenodo.18727694 (DOI)
Dates
- Issued
-
2026-02-28
References
- Anthropic. (2025). Context windows. https://docs.anthropic.com/en/docs/build-with-claude/context-windows
- BuiltWith. (n.d.). LLMS Text usage statistics. Accessed February 2026. https://trends.builtwith.com/robots/LLMS-Text
- Cloudflare. (2026, February 12). Markdown for Agents: Let AI read your site, not scrape it. https://blog.cloudflare.com/markdown-for-agents/
- Google. (2025). Long context. Vertex AI Generative AI Documentation. https://docs.cloud.google.com/vertex-ai/generative-ai/docs/long-context
- Howard, J. (2024, September 3). llms.txt: A proposal for LLM-friendly website content. https://llmstxt.org/
- Howard, J., & Ruder, S. (2018). Universal Language Model Fine-tuning for Text Classification (ULMFiT). Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. https://arxiv.org/abs/1801.06146
- Karpathy, A. (2025). Commentary on LLM attention share. As reported in Shrivastava, R. (2025, June 19). Forbes. https://www.forbes.com/sites/rashishrivastava/2025/06/19/these-startups-are-helping-businesses-show-up-in-ai-search-summaries/
- Mueller, J. (2026). As reported in Search Engine Journal, February 4, 2026. https://www.searchenginejournal.com/googles-mueller-calls-markdown-for-bots-idea-a-stupid-idea/566598/
- OpenAI. (2026). GPT-5 model documentation. https://platform.openai.com/docs/models/gpt-5/
- Rankability. (2025). LLMS.txt adoption research report 2025. https://www.rankability.com/data/llms-txt-adoption/
- Winum, B. (2025, July 2). The Death of Traditional Digital Marketing. https://brianwinum.com/the-death-of-traditional-digital-marketing-why-ai-demands-a-new-push-pull-strategy/
- Holmes, G. C. (2026). Agentic Recommendation Optimization: A Blueprint for the Post-Search Economy. Zenodo. https://doi.org/10.5281/zenodo.18727694