How to create llms.txt

If you have not decided whether you need the file at all: What is llms.txt.

1. Pick the main pages.

   Picture a person who arrives at the site for the first time with a question. Which five to twenty pages answer most questions? That is the content of the file.

   Leave out: login and cart pages, search results, service and private sections, duplicates.

2. Group them by meaning.

   Two to five sections are enough for almost any site: documentation and examples, catalog and terms, articles and help.

3. Write the file.

   The site name, one or two sentences about it, then sections with links. Each link gets a note saying what is behind it.

4. Put the file at the site root.

   It has to open at https://your-site/llms.txt and be served as text.

5. Check the server response.

   Status 200, a text content type, the first line starts with a hash sign. A common mistake: the site serves an HTML page instead of the file.

6. Decide how the file will be refreshed.

   A file that does not change together with the site points to missing pages half a year later.

The same file as a whole:

# Kolos API

> A delivery service for online stores: rate calculation, order creation, tracking.

Every request needs a key. Test environment: sandbox.kolos.example

## Documentation

- [Quick start](https://kolos.example/docs/start.md): the key, the first request, the first order
- [API reference](https://kolos.example/docs/api.md): methods, parameters, error codes

## Optional

- [Changelog](https://kolos.example/changelog.md): versions and dates

Exact rules: the file format, sections. More samples: basic examples.

The file goes to the site root, next to robots.txt. For a static site that is the folder with public files. For a site on a content management system an extension usually serves the file, and it does not exist on disk.

The second edition of the proposal also allows a file in a section of the site. It then describes only the addresses inside that section:

/llms.txt        the whole site
/docs/llms.txt   the /docs/ section only

curl -sI https://example.com/llms.txt
curl -s https://example.com/llms.txt | head -n 3

The full list of checks: Validating files and markup.

The second edition added a relation through which a page says which llms.txt describes it:

<link rel="describedby" href="/llms.txt">

This step is optional. More: Discovery of the file and Markdown versions.

When the site runs on a content management system, there is no reason to write the file by hand: an extension assembles it from existing pages.

An automatically assembled file is checked the same way as one written by hand. It is also worth reading through: an extension knows the structure of the site but not what matters on it.

• Do not copy the sitemap. The full list of addresses is already in sitemap.xml. This file needs selection.
• Do not write prohibitions. The file blocks nothing. Robot access is configured in robots.txt.
• Do not add private things. The file is public: no admin addresses, no drafts.
• Do not expect better rankings. Search engines do not use the file for ranking: Does llms.txt work.

• Best practices - how to make the file useful
• An AI-ready site - the other layers: access, markup, content
• What is llms-full.txt - whether a second file is needed
• A prompt for generation - a draft of the file with the help of an AI system