# llms.txt and AI search blog

> The chronicle of 2026: what happened, what the studies measured and what it means for a site owner

Page: https://llms-full-txt.ru/en/blog/
Site index: https://llms-full-txt.ru/en/llms.txt

The chronicle of the ecosystem: what happened, what was measured and what it means for a site owner. Notes are ordered by the date of the event. Evergreen material lives in the guides, and every note links to the page that is kept up to date.

## Chronicle

- analysis October 3, 2026

### [llms.txt in autumn 2026: what is known](https://llms-full-txt.ru/en/blog/state-of-llms-txt-autumn-2026/)

A summary of the year - the second edition of the proposal, the Lighthouse check, Google’s position, studies of requests and file quality. What is confirmed, what is not proven and what to do with a site

Kept up to date in: [llms.txt status](https://llms-full-txt.ru/en/guides/standard-status/)

- news event September 18, 2026 published October 3, 2026

### [Lighthouse 13.5: llms.txt becomes one of the discovery points](https://llms-full-txt.ru/en/blog/lighthouse-13-5-agent-discovery/)

In Lighthouse 13.5.0 the llms.txt check is grouped with other ways an agent discovers a site - what changed in the report and what stayed the same

Kept up to date in: [Lighthouse and llms.txt](https://llms-full-txt.ru/en/guides/lighthouse/)

- study event September 16, 2026 published October 3, 2026

### [Mintlify: agents request documentation twice as often as people](https://llms-full-txt.ru/en/blog/mintlify-agents-outnumber-people/)

A documentation platform’s report for August 2026 - 257 million agent requests against 131 million human page loads. What it says about documentation and what it does not say about other sites

Kept up to date in: [Markdown versions of pages](https://llms-full-txt.ru/en/guides/markdown-pages/)

- news event September 14, 2026 published October 3, 2026

### [IETF AIPREF: the AI preference vocabulary is still a draft](https://llms-full-txt.ru/en/blog/ietf-aipref-drafts/)

The IETF AIPREF working group updated the draft vocabulary a site can use to say whether its content may be used for training and answers - what is in it and why it is too early to rely on it

Kept up to date in: [AI crawlers and robots.txt](https://llms-full-txt.ru/en/guides/ai-crawlers/)

- study event August 31, 2026 published October 3, 2026

### [Common Crawl analyzed 584 thousand llms.txt files](https://llms-full-txt.ru/en/blog/common-crawl-584k-llms-txt/)

The first analysis of llms.txt content from an open web crawl - 68% of files made by a generator, 23% without a single link, and 40% of successful responses turned out to be an HTML page

Kept up to date in: [Best practices](https://llms-full-txt.ru/en/guides/best-practices/)

- news event August 10, 2026 published October 3, 2026

### [The second edition of llms.txt is out](https://llms-full-txt.ru/en/blog/llms-txt-v2/)

On August 10, 2026, the second edition of the llms.txt proposal was published - discovery relations, files in subpaths, two forms of Markdown addresses and a new role for the Optional section

Kept up to date in: [Changes in v2](https://llms-full-txt.ru/en/specification/changes/)

- study event July 17, 2026 published October 3, 2026

### [Mintlify test: with llms.txt an agent hits fewer missing pages](https://llms-full-txt.ru/en/blog/mintlify-markdown-benchmark/)

Mintlify compared four documentation formats on 20 sites and 2,400 runs - what changed, what stayed the same and why the source has to be treated as an interested party

Kept up to date in: [Markdown versions of pages](https://llms-full-txt.ru/en/guides/markdown-pages/)

- news event June 22, 2026 published October 3, 2026

### [Chrome describes how to prepare a site for agents](https://llms-full-txt.ru/en/blog/chrome-agent-ready-toolkit/)

The Chrome blog published an agent-ready toolkit - the Agentic Browsing category in Lighthouse, WebMCP and DevTools for agents. The post does not mention llms.txt

Kept up to date in: [Lighthouse and llms.txt](https://llms-full-txt.ru/en/guides/lighthouse/)

- study event June 15, 2026 published October 3, 2026

### [Ahrefs: 97% of llms.txt files were never requested](https://llms-full-txt.ru/en/blog/ahrefs-llms-txt-study/)

The Ahrefs study of 137 thousand domains - how many sites publish llms.txt, who requests it and why the numbers need to be read with caveats

Kept up to date in: [Does llms.txt work](https://llms-full-txt.ru/en/guides/does-llms-txt-work/)

- analysis event June 15, 2026 published October 3, 2026

### [Google documents it: Search does not use llms.txt](https://llms-full-txt.ru/en/blog/google-search-ignores-llms-txt/)

What exactly the Google guide on AI features in Search says, what it covers and what it does not, and how it fits with the Lighthouse check

Kept up to date in: [Does llms.txt work](https://llms-full-txt.ru/en/guides/does-llms-txt-work/)

- news event May 7, 2026 published October 3, 2026

### [Lighthouse starts checking llms.txt](https://llms-full-txt.ru/en/blog/lighthouse-checks-llms-txt/)

In Lighthouse 13.3.0 the llms.txt check is on by default - what exactly it looks at, why an empty file passes and what it does not prove

Kept up to date in: [Lighthouse and llms.txt](https://llms-full-txt.ru/en/guides/lighthouse/)

- news March 1, 2026 updated October 3, 2026

### [llms.txt in 2026: From Experiment to Standard](https://llms-full-txt.ru/en/blog/llms-txt-2026/)

How llms.txt evolved from Jeremy Howard's idea to 10% adoption among 300K domains. What changed and what's next.

Kept up to date in: [llms.txt status](https://llms-full-txt.ru/en/guides/standard-status/)

- guide February 15, 2026 updated October 3, 2026

### [AI crawlers guide: who to allow, who to block](https://llms-full-txt.ru/en/blog/ai-crawlers-guide/)

Complete guide to AI crawlers in 2026 — GPTBot, ClaudeBot, Google-Extended. How to configure robots.txt for AI bots.

Kept up to date in: [AI crawlers and robots.txt](https://llms-full-txt.ru/en/guides/ai-crawlers/)

- news event February 12, 2026 published October 3, 2026

### [Cloudflare serves Markdown to agents instead of HTML](https://llms-full-txt.ru/en/blog/cloudflare-markdown-for-agents/)

Cloudflare turned on page-to-Markdown conversion by the Accept header - how it works, how much it saves and what stays on the site owner’s side

Kept up to date in: [Markdown versions of pages](https://llms-full-txt.ru/en/guides/markdown-pages/)

- study event January 15, 2026 published October 3, 2026

### [Web Almanac: two sites in a hundred have llms.txt](https://llms-full-txt.ru/en/blog/web-almanac-2025-llms-txt/)

The SEO chapter of Web Almanac 2025 counted llms.txt across the HTTP Archive crawl for the first time - 2.1% of sites, and most files come from WordPress plugins

Kept up to date in: [Who publishes llms.txt](https://llms-full-txt.ru/en/reference/adoption/)

[LLMs Full Text](https://llms-full-txt.ru/en/) A reference on llms.txt and AI-ready sites: format, examples, tools.

Start here

- [An AI-ready site](https://llms-full-txt.ru/en/guides/ai-ready-site/)
- [What is llms.txt](https://llms-full-txt.ru/en/guides/what-is-llms-txt/)
- [Getting Started](https://llms-full-txt.ru/en/guides/getting-started/)
- [Best Practices](https://llms-full-txt.ru/en/guides/best-practices/)
- [Status of llms.txt](https://llms-full-txt.ru/en/guides/standard-status/)

llms.txt

- [What is llms-full.txt](https://llms-full-txt.ru/en/guides/llms-full-txt/)
- [llms.txt file format](https://llms-full-txt.ru/en/specification/format/)
- [Sections and Structure](https://llms-full-txt.ru/en/specification/sections/)
- [Discovering the file and Markdown versions](https://llms-full-txt.ru/en/specification/discovery/)
- [Changes in v2](https://llms-full-txt.ru/en/specification/changes/)
- [Basic Examples](https://llms-full-txt.ru/en/examples/basic/)
- [Advanced Examples](https://llms-full-txt.ru/en/examples/advanced/)

A machine-readable site

- [Structured data and JSON-LD](https://llms-full-txt.ru/en/guides/json-ld/)
- [Markdown versions of pages](https://llms-full-txt.ru/en/guides/markdown-pages/)
- [AI crawlers and robots.txt](https://llms-full-txt.ru/en/guides/ai-crawlers/)
- [Validating files and markup](https://llms-full-txt.ru/en/guides/validation/)
- [Lighthouse and llms.txt](https://llms-full-txt.ru/en/guides/lighthouse/)
- [llms.txt in WordPress](https://llms-full-txt.ru/en/guides/wordpress/)
- [llms.txt and JSON-LD in Webasyst](https://llms-full-txt.ru/en/guides/webasyst/)

AI and search

- [AI Search Optimization](https://llms-full-txt.ru/en/guides/geo-aeo/)
- [How to measure AI visibility](https://llms-full-txt.ru/en/guides/ai-visibility/)
- [Does llms.txt work](https://llms-full-txt.ru/en/guides/does-llms-txt-work/)

Tools

- [Generators and Tools](https://llms-full-txt.ru/en/tools/generators/)
- [llm.codes](https://llms-full-txt.ru/en/tools/llm-codes/)
- [llmstxt-architect](https://llms-full-txt.ru/en/tools/llmstxt-architect/)
- [llmstxt CLI](https://llms-full-txt.ru/en/tools/llmstxt-cli/)

Reference

- [Setup in documentation generators](https://llms-full-txt.ru/en/reference/configuration/)
- [llms.txt and MCP](https://llms-full-txt.ru/en/reference/mcp-relationship/)
- [Who publishes llms.txt](https://llms-full-txt.ru/en/reference/adoption/)
- [Glossary](https://llms-full-txt.ru/en/reference/glossary/)
- [Paying and working from Russia](https://llms-full-txt.ru/en/reference/from-russia/)

Prompts

- [All prompts](https://llms-full-txt.ru/en/prompts/)
- [Generate llms.txt](https://llms-full-txt.ru/en/prompts/generate-llms-txt/)
- [Audit llms.txt](https://llms-full-txt.ru/en/prompts/audit-llms-txt/)
- [Optimize content for AI](https://llms-full-txt.ru/en/prompts/optimize-for-ai/)

Project

- [Blog](https://llms-full-txt.ru/en/blog/)
- [llms.txt](https://llms-full-txt.ru/en/llms.txt)
- [llms-full.txt](https://llms-full-txt.ru/en/llms-full.txt)
- [RSS](https://llms-full-txt.ru/en/rss.xml)

© 2025-2026 LLMs Full Text

Built by