# llms.txt reference: format, examples and tools

> llms.txt is a file at the site root that lists a site's main pages for AI agents. A reference on the v2 format, examples, tools, validation and the evidence on whether it works

Page: https://llms-full-txt.ru/en/
Site index: https://llms-full-txt.ru/en/llms.txt

Search is changing: more and more people get the answer right inside ChatGPT, Gemini and other AI systems. This reference explains how to prepare a site for that world - from content structure and Markdown to llms.txt, crawlers and GEO.

[What to do with my site](https://llms-full-txt.ru/en/guides/ai-ready-site/)

Who reads a site now

- A person asks a question
- AI search or an agent looks for a source
- Your site serves content in a clear form HTML Markdown llms.txt structured data
- The answer with a link to the source

## Your site is now read not only by people

What has changed

Before

- A person types a query
- Search shows a list of links
- A site the person reads the page

Now

- A person asks a question
- AI search or an agent finds and reads pages by itself
- Sources several sites at once, yours among them
- A ready answer the person may not open a single page

A site used to be prepared for a browser and a search robot first. Now there is one more consumer: the AI agent. It also has to find the right page, understand its structure and get the content without noise.

## Where is llms.txt here?

A map of material for an AI agent

llms.txt is a file in the site root: the name, a short description and lists of links to what matters.

[File format](https://llms-full-txt.ru/en/specification/format/)

Three files, three different questions

- `robots.txt` where a robot may go
- `sitemap.xml` which pages exist
- `llms.txt` which material may be useful to an agent

This is a simplified model, not three interchangeable standards.

## One file is not enough to make a site ready for AI

A site ready for AI

Five parts of one task. llms.txt covers one of them.

- [Find](https://llms-full-txt.ru/en/guides/ai-crawlers/) Crawlers, robots.txt, links, indexability.
- [Understand](https://llms-full-txt.ru/en/guides/markdown-pages/) Clear page structure, headings, Markdown.
- [Choose](https://llms-full-txt.ru/en/specification/format/) llms.txt, docs navigation, files in subpaths.
- [Trust](https://llms-full-txt.ru/en/guides/does-llms-txt-work/) Sources, authorship, freshness, evidence.
- [Measure](https://llms-full-txt.ru/en/guides/ai-visibility/) Bot logs, AI referrals, mentions and citations.

## Does it help promotion at all?

What is known and what is not

Confirmed

- Large documentation sites publish llms.txt themselves.
- Agents do work with Markdown versions of pages.
- Lighthouse checks the file in the experimental Agentic Browsing category.
- AI bots visit sites: their owners describe it in their own documentation.

Not proven

- That having llms.txt improves search rankings.
- That the file guarantees a place in a ChatGPT answer.
- That it increases citations by itself.

llms.txt is one part of preparing a site for AI, not a magic SEO button.

[What is known about the benefit](https://llms-full-txt.ru/en/guides/does-llms-txt-work/)

## What “promotion in neural networks” means in practice

- GEO Generative Engine Optimization
- AEO Answer Engine Optimization
- AI SEO SEO for AI search
- AI visibility being visible in AI

One task

To be found, understood and used by an AI system

What AI visibility is made of

- Accessibility the page can be fetched
- Understanding the structure can be parsed
- Citable content there is something to quote
- Measurement you can see what happens

Many terms, one meaning. Nobody can promise a place in an AI answer.

[GEO and AEO without promises](https://llms-full-txt.ru/en/guides/geo-aeo/)

## A site can be prepared step by step

Check access

AI bots can fetch the pages of the site.

[AI crawlers](https://llms-full-txt.ru/en/guides/ai-crawlers/)

Tidy the content

Clear headings and page structure.

[Best practices](https://llms-full-txt.ru/en/guides/best-practices/)

Add Markdown

Where it is justified: documentation, help.

[Markdown versions of pages](https://llms-full-txt.ru/en/guides/markdown-pages/)

Create llms.txt

A correct file by the second edition.

[Getting started](https://llms-full-txt.ru/en/guides/getting-started/)

Check the file and links

Status code, structure, live addresses.

[Checklist](https://llms-full-txt.ru/en/guides/validation/)

Measure the result

Bot logs, AI referrals, mentions and citations.

[Measurement](https://llms-full-txt.ru/en/guides/ai-visibility/)

## You do not have to assemble everything by hand

For some CMS there are ready-made extensions: they collect the site structure, build llms.txt and keep it up to date.

How a ready-made solution works

- CMS pages, products, posts
- Extension reads what is published
- Site structure sections and links
- llms.txt built and kept fresh
- Check the same as for a hand-written file
- Webasyst

Site

A map of site pages

Data source

Published site pages, the site name and contacts

Creates

`llms.txt``llms-full.txt`

ready-made The base edition is free

[View the extension](https://www.webasyst.com/store/plugin/site/kaktak_llmstxt/?promo_id=BE462B3B6D)

- Webasyst

Shop-Script

A map of an online store

Data source

Categories, products, brands, information pages and contacts

Creates

`llms.txt``llms-full.txt`

ready-made

[View the extension](https://www.webasyst.com/store/plugin/shop/kaktak_llmstxt/?promo_id=BE462B3B6D)

- Webasyst

Blog

A map of blog posts

Data source

Published posts, categories and tags

Creates

`llms.txt``llms-full.txt`

ready-made

[View the extension](https://www.webasyst.com/store/plugin/blog/kaktak_llmstxt/?promo_id=BE462B3B6D)

- Webasyst

Hub

A map of community discussions

Data source

Public sections and published discussions

Creates

`llms.txt``llms-full.txt`

ready-made

[View the extension](https://www.webasyst.com/store/plugin/hub/kaktak_llmstxt/?promo_id=BE462B3B6D)

- Webasyst

Helpdesk

A map of a knowledge base

Data source

Public FAQ and pages; tickets and correspondence are left out

Creates

`llms.txt``llms-full.txt`

ready-made

[View the extension](https://www.webasyst.com/store/plugin/helpdesk/kaktak_llmstxt/?promo_id=BE462B3B6D)

- Webasyst

Photos

A map of photo albums

Data source

Public albums and pages

Creates

`llms.txt``llms-full.txt`

ready-made

[View the extension](https://www.webasyst.com/store/plugin/photos/kaktak_llmstxt/?promo_id=BE462B3B6D)

Promo code for a discount on your first Webasyst order: BE462B3B6D

Commercial extensions from the Webasyst Store. Each one builds llms.txt and llms-full.txt from the data of its app and rebuilds them on a schedule or on demand. A file made by an extension is checked the same way as one written by hand.

## Where to start for you

- Site owner

«I need to understand whether I should do anything at all»

- [A plain explanation](https://llms-full-txt.ru/en/guides/what-is-llms-txt/)
- [What is proven and what is not](https://llms-full-txt.ru/en/guides/does-llms-txt-work/)
- [How to check my site](https://llms-full-txt.ru/en/guides/validation/)
- SEO specialist and marketer

«I want to understand GEO, AI crawlers and measurement»

- [GEO and AEO](https://llms-full-txt.ru/en/guides/geo-aeo/)
- [AI crawlers and robots.txt](https://llms-full-txt.ru/en/guides/ai-crawlers/)
- [Status and evidence](https://llms-full-txt.ru/en/guides/does-llms-txt-work/)
- Developer

«I want to implement it correctly»

- [Format by the second edition](https://llms-full-txt.ru/en/specification/format/)
- [What changed in v2](https://llms-full-txt.ru/en/specification/changes/)
- [Advanced scenarios](https://llms-full-txt.ru/en/examples/advanced/)

## It is already used in real projects

These companies publish llms.txt or Markdown representations in their own documentation. It does not mean their agents read your file.

- [Anthropic docs. anthropic. com](https://docs.anthropic.com/llms.txt)
- [Cloudflare developers. cloudflare. com](https://developers.cloudflare.com/llms.txt)
- [Vercel vercel. com](https://vercel.com/llms.txt)
- [Yandex yandex. cloud](https://yandex.cloud/llms.txt)
- [Stripe docs. stripe. com](https://docs.stripe.com/llms.txt)
- [Supabase supabase. com](https://supabase.com/llms.txt)

llms.txt files found in an open web archive

584K

Common Crawl, August 2026

of sampled sites publish the file

2.1%

Web Almanac 2025

the second edition of the proposal

v2

llmstxt.org, August 10, 2026

[Real files explained](https://llms-full-txt.ru/en/examples/advanced/)

## See how it works

- [llms.txt in five minutes](https://llms-full-txt.ru/en/guides/getting-started/) A minimal file, where to put it and how to check it.
- Write the file
- Put it in the site root
- Check the server response
- [llms.txt and llms-full.txt](https://llms-full-txt.ru/en/guides/llms-full-txt/) An index and the full text: the difference.

`llms.txt` an index of links `llms-full.txt` all text in one file

- [AI crawlers](https://llms-full-txt.ru/en/guides/ai-crawlers/) Three kinds of bots, three different robots.txt decisions.

training search user request

- [llms.txt and MCP](https://llms-full-txt.ru/en/reference/mcp-relationship/) Two independent systems and a bridge.

a file a protocol

- [Real files](https://llms-full-txt.ru/en/examples/advanced/) How OpenAI, Stripe and others build theirs.
- [Markdown for agents](https://llms-full-txt.ru/en/guides/markdown-pages/) A clean version of a page at the same address.
- [Discovery in v2](https://llms-full-txt.ru/en/specification/discovery/) The alternate and describedby relations.
- [Does llms.txt work](https://llms-full-txt.ru/en/guides/does-llms-txt-work/) Studies and their limits.

## What is changing right now

- September 18, 2026 event

Lighthouse 13.5.0 grouped the llms.txt check with other agent discovery points

For a site owner. The check stays in the report, now next to other ways an agent discovers a site.

[Read the note](https://llms-full-txt.ru/en/blog/lighthouse-13-5-agent-discovery/)

- August 31, 2026 study

Common Crawl analyzed 584 thousand files: 68% made by a generator, 23% without a single link

For a site owner. A file without links is useless: check the content, not just the presence.

[Read the note](https://llms-full-txt.ru/en/blog/common-crawl-584k-llms-txt/)

- August 10, 2026 edition

The second edition of the proposal: discovery relations and files in subpaths

For a site owner. The file can be declared in the page code, and sections can have files of their own.

[Read the note](https://llms-full-txt.ru/en/blog/llms-txt-v2/)

- June 15, 2026 study

Ahrefs: 97% of the files were never requested

For a site owner. The file does not bring bots by itself: read the server logs.

[Read the note](https://llms-full-txt.ru/en/blog/ahrefs-llms-txt-study/)

- June 15, 2026 event

Google documented it: Search does not use such files

For a site owner. Do not expect an effect on Google rankings.

[Read the note](https://llms-full-txt.ru/en/blog/google-search-ignores-llms-txt/)

- May 7, 2026 event

Lighthouse 13.3.0 started checking llms.txt by default

For a site owner. A missing file became visible in a regular page report.

[Read the note](https://llms-full-txt.ru/en/blog/lighthouse-checks-llms-txt/)

- October 3, 2026 [Ahrefs: 97% of llms.txt files were never requested](https://llms-full-txt.ru/en/blog/ahrefs-llms-txt-study/)
- October 3, 2026 [Chrome describes how to prepare a site for agents](https://llms-full-txt.ru/en/blog/chrome-agent-ready-toolkit/)
- October 3, 2026 [Cloudflare serves Markdown to agents instead of HTML](https://llms-full-txt.ru/en/blog/cloudflare-markdown-for-agents/)
- [All blog posts](https://llms-full-txt.ru/en/blog/)

## Need to prepare an existing site?

Audit and implementation are done by the kaktak.net team.

[Discuss the task](https://kaktak.net/)

[LLMs Full Text](https://llms-full-txt.ru/en/) A reference on llms.txt and AI-ready sites: format, examples, tools.

Start here

- [An AI-ready site](https://llms-full-txt.ru/en/guides/ai-ready-site/)
- [What is llms.txt](https://llms-full-txt.ru/en/guides/what-is-llms-txt/)
- [Getting Started](https://llms-full-txt.ru/en/guides/getting-started/)
- [Best Practices](https://llms-full-txt.ru/en/guides/best-practices/)
- [Status of llms.txt](https://llms-full-txt.ru/en/guides/standard-status/)

llms.txt

- [What is llms-full.txt](https://llms-full-txt.ru/en/guides/llms-full-txt/)
- [llms.txt file format](https://llms-full-txt.ru/en/specification/format/)
- [Sections and Structure](https://llms-full-txt.ru/en/specification/sections/)
- [Discovering the file and Markdown versions](https://llms-full-txt.ru/en/specification/discovery/)
- [Changes in v2](https://llms-full-txt.ru/en/specification/changes/)
- [Basic Examples](https://llms-full-txt.ru/en/examples/basic/)
- [Advanced Examples](https://llms-full-txt.ru/en/examples/advanced/)

A machine-readable site

- [Structured data and JSON-LD](https://llms-full-txt.ru/en/guides/json-ld/)
- [Markdown versions of pages](https://llms-full-txt.ru/en/guides/markdown-pages/)
- [AI crawlers and robots.txt](https://llms-full-txt.ru/en/guides/ai-crawlers/)
- [Validating files and markup](https://llms-full-txt.ru/en/guides/validation/)
- [Lighthouse and llms.txt](https://llms-full-txt.ru/en/guides/lighthouse/)
- [llms.txt in WordPress](https://llms-full-txt.ru/en/guides/wordpress/)
- [llms.txt and JSON-LD in Webasyst](https://llms-full-txt.ru/en/guides/webasyst/)

AI and search

- [AI Search Optimization](https://llms-full-txt.ru/en/guides/geo-aeo/)
- [How to measure AI visibility](https://llms-full-txt.ru/en/guides/ai-visibility/)
- [Does llms.txt work](https://llms-full-txt.ru/en/guides/does-llms-txt-work/)

Tools

- [Generators and Tools](https://llms-full-txt.ru/en/tools/generators/)
- [llm.codes](https://llms-full-txt.ru/en/tools/llm-codes/)
- [llmstxt-architect](https://llms-full-txt.ru/en/tools/llmstxt-architect/)
- [llmstxt CLI](https://llms-full-txt.ru/en/tools/llmstxt-cli/)

Reference

- [Setup in documentation generators](https://llms-full-txt.ru/en/reference/configuration/)
- [llms.txt and MCP](https://llms-full-txt.ru/en/reference/mcp-relationship/)
- [Who publishes llms.txt](https://llms-full-txt.ru/en/reference/adoption/)
- [Glossary](https://llms-full-txt.ru/en/reference/glossary/)
- [Paying and working from Russia](https://llms-full-txt.ru/en/reference/from-russia/)

Prompts

- [All prompts](https://llms-full-txt.ru/en/prompts/)
- [Generate llms.txt](https://llms-full-txt.ru/en/prompts/generate-llms-txt/)
- [Audit llms.txt](https://llms-full-txt.ru/en/prompts/audit-llms-txt/)
- [Optimize content for AI](https://llms-full-txt.ru/en/prompts/optimize-for-ai/)

Project

- [Blog](https://llms-full-txt.ru/en/blog/)
- [llms.txt](https://llms-full-txt.ru/en/llms.txt)
- [llms-full.txt](https://llms-full-txt.ru/en/llms-full.txt)
- [RSS](https://llms-full-txt.ru/en/rss.xml)

© 2025-2026 LLMs Full Text

Built by