Standard Status
llms.txt status in 2026 — from 15 sites to 10% adoption among 300K domains. Skeptics vs proponents arguments, three development scenarios
Origin
Section titled “Origin”Jeremy Howard (Answer.AI / fast.ai) proposed llms.txt in September 2024. The idea: give LLMs a structured Markdown file instead of forcing them to parse HTML.
llms.txt is not a formal standard. No W3C or IETF process. It’s a community-driven convention, similar to robots.txt in 1994 — de facto first, then (28 years later) RFC 9309.
Repository AnswerDotAI/llms-txt: 2,191 stars, 118 forks, 65 open issues, Apache-2.0 license (as of January 2026).
Adoption Timeline
Section titled “Adoption Timeline”| Period | Number of Sites | Source |
|---|---|---|
| February 2025 | ~15 | early adopters |
| May 2025 | ~105 | llms-txt-hub |
| October 2025 | 844,000+ | ALLMO.ai analysis |
| January 2026 | 10.13% of 300K domains | SE Ranking |
The llms-txt-hub directory (732 stars, 381 forks) is the largest llms.txt site catalog.
Who Uses It
Section titled “Who Uses It”| Category | Examples |
|---|---|
| AI/ML | Anthropic, Hugging Face, Cohere, CrewAI, LangChain |
| DevTools | Cloudflare, Stripe, Vercel, Astro, Resend |
| CMS/No-code | Yoast, Zapier, Make, Activepieces |
| Documentation | MDN, LangChain, Supabase, Turso |
| Databases | DuckDB, Upstash, Turso |
The Debate
Section titled “The Debate”Adoption is growing, but evidence of direct impact on AI citations remains mixed.
Skeptic Arguments
Section titled “Skeptic Arguments”John Mueller (Google), June 2025:
“No AI system uses llms.txt to generate answers.”
OtterlyAI (90-day experiment): 0.1% of AI crawler requests went to /llms.txt — 84 visits over 3 months vs 265 average per page. AI bots do visit, but rarely.
SE Ranking (2026): analyzing 300K domains found no correlation between having llms.txt and AI citation frequency.
Proponent Arguments
Section titled “Proponent Arguments”Vercel: 10% of new signups come from ChatGPT. Structured content works.
Dev5310: Google AI Mode cited a site with llms.txt on day one after publishing.
Ecosystem: 14+ generators, plugins for WordPress (5+), MkDocs, Docusaurus, Nuxt, Astro, Discourse. Developers are voting with code.
The Core Disagreement
Section titled “The Core Disagreement”Skeptics evaluate llms.txt as a standalone signal for AI search. Proponents see broader use cases: MCP, RAG, developer experience, AI assistants in IDEs.
Three Scenarios
Section titled “Three Scenarios”1. Gradual Adoption (~40%)
Section titled “1. Gradual Adoption (~40%)”AI providers (OpenAI, Google, Anthropic) begin formally supporting llms.txt. The file becomes a standard part of the SEO stack — like sitemap.xml.
2. Evolution into MCP (~30%)
Section titled “2. Evolution into MCP (~30%)”Model Context Protocol (Anthropic → Linux Foundation) absorbs llms.txt use cases. The file remains as a lightweight discovery layer for MCP servers.
3. Decline (~30%)
Section titled “3. Decline (~30%)”AI models become smart enough to parse any content. llms.txt remains a niche solution for documentation.
Recommendation
Section titled “Recommendation”Creating llms.txt is a low risk, low cost decision:
- Time: 15-30 minutes for manual creation, 0 minutes with auto-generation
- Risks: no negative side effects
- Value even without AI search: MCP integration, RAG pipelines, context for AI assistants, structured documentation
If you maintain documentation or a technical site — implementation is justified. Cost is minimal, potential is significant.
Sources: