Status of llms.txt

• llms.txt is a proposal by a single author, not an IETF or W3C standard.
• v2 was published on August 10, 2026.
• Documentation platforms and AI companies publish the file at scale, for their own developers.
• There is no evidence that the file improves search rankings or citation frequency.
• No bot owner states that its bot reads other sites’ /llms.txt.

Jeremy Howard (Answer.AI) published the proposal on September 3, 2024. No formal process stands behind it: no working group, no RFC. The text and the discussions live in the AnswerDotAI/llms-txt repository.

That is why this site says “proposal” or “proposed standard” rather than just “standard”.

As of October 3, 2026 the repository has 2,643 stars and 108 open issues. The license is Apache-2.0.

Date	Event
2024-09-03	v1 of the proposal
2026-05-01	the llms.txt audit is added in Lighthouse 13.2.0
2026-05-07	the audit is on by default in Lighthouse 13.3.0
2026-08-10	v2 of the proposal

What changed in v2 is on the Changes in v2 page.

We checked 46 sites by requesting /llms.txt on October 3, 2026.

Publish it: OpenAI, Anthropic, Cloudflare, Vercel, Stripe, Supabase, GitHub, Cursor, Perplexity, LangChain, Docker, React, Vue, Svelte, Shopify, Yoast, Wix, Gravity UI, Yandex Cloud.

Generate it automatically: the documentation platforms Mintlify and GitBook, the Yoast SEO and AIOSEO plugins for WordPress, the Wix site builder.

Do not publish it (404): docs.astro.build, MDN, the Hugging Face root, developer.chrome.com, web.dev, Tailwind, Ahrefs, Moz, habr.com, Tilda, 1C-Bitrix.

The list changes. Older roundups, including the previous version of this page, name sites that have no file.

Source	What was measured	Result
HTTP Archive, Web Almanac 2025	share of sites with the file	2.1%
Common Crawl, August 2026	files found	584,107
Common Crawl, same analysis	created by a generator	68%
Common Crawl, same analysis	hosted on Wix	41%
Common Crawl, same analysis	without a single link	23%

Most files are generated automatically rather than written by the site owner. The number of files says little about how many people made a deliberate decision.

The full story of the audit: Lighthouse and llms.txt.

Lighthouse has an audit for the presence of llms.txt. It belongs to the experimental Agentic Browsing category.

• Only the availability of the file is checked. The content is not evaluated.
• The category has no 0-100 score.
• If the file answers 404, the audit is reported as not applicable rather than failed.

• Coding agents request documentation through llms.txt and Markdown versions of pages. Documentation platforms report this; they also have a stake in the subject.
• AI companies publish such files for their own documentation. So the format is considered useful for developer tasks.

• Effect on citations. SE Ranking studied 300,000 domains and found no relationship between having the file and citation frequency in AI answers.
• Requests for the file. Ahrefs studied 137,000 domains: 97% of the files were never requested.
• Search bots reading the file. Neither OpenAI, Anthropic, Google nor Perplexity state that their bots read other sites’ /llms.txt.

llms.txt is an interface to documentation and content for agents. As a lever for search visibility it is unconfirmed. A promise of higher rankings or more citations from the file alone has no evidence behind it.

The proposal repository is discussing:

• which language the root file of a multilingual site should use;
• which Content-Type to serve;
• a section on security and untrusted content;
• signing and provenance of the file;
• what to do when /llms.txt answers 200 with an HTML page.

The last one is not hypothetical. In the Common Crawl corpus, 40% of successful responses to /llms.txt were HTML.

Yes, if you have documentation, an API, a reference or other material that agents and AI assistants in code editors work with. The cost is low, and many platforms generate the file for you.

Not for this reason, if the only goal is search rankings or mentions in AI answers. There is no data showing the file affects them.

In any case: the file does not control access. Rules for bots belong in robots.txt - see AI crawlers and robots.txt.