# llms.txt validator and checker: files and markup

> How to check llms.txt, llms-full.txt, Markdown versions of pages and structured data: status code, content type, structure, links, validators and typical mistakes

Page: https://llms-full-txt.ru/en/guides/validation/
Site index: https://llms-full-txt.ru/en/llms.txt

A check answers three questions: the file reaches the client, it is built correctly, and what it says matches the site.

A file made by an extension or a generator is checked the same way as one written by hand. Automatic assembly guarantees neither the right address nor working links.

A command line and two free validators are enough.

## Why this is not a formality

[Common Crawl](https://commoncrawl.org/blog) analyzed 584 thousand `llms.txt` files found on the web. A large share of them does not work as intended.

- 40%
- of successful responses at /llms.txt were an HTML page: 23%
- of files contain no links at all: 50%
- of files have the full structure: title, summary, sections

The most common failure is invisible to the owner: the site answers `/llms.txt` with `200` but serves the home page or a "not found" page.

## Checking llms.txt

1. `https://site/llms.txt` answers `200` with no redirect chain.
2. The content type is `text/plain` or `text/markdown`, UTF-8.
3. The body is text, not HTML. The first line starts with `# `.
4. There is exactly one first-level heading.
5. A short summary follows the heading; the line starts with `> `.
6. Links are written as `- [Title](address): note`.
7. Link addresses are absolute and answer `200`.
8. The file lists no private pages, drafts or addresses with parameters.
9. The file stays a reasonable size: it is a map, not the whole site.
10. `robots.txt` does not block the file from the robots it is meant for.

```bash
curl -sI https://example.com/llms.txt
curl -s https://example.com/llms.txt | head -n 5
```

In the first command look at the `HTTP` and `content-type` lines. In the second, the first line has to start with a hash sign.

Check every link in the file at once:

```bash
curl -s https://example.com/llms.txt \
  | grep -oE '\]\(https?://[^)]+\)' | tr -d '()]' \
  | while read -r url; do
      printf '%s %s\n' "$(curl -s -o /dev/null -w '%{http_code}' "$url")" "$url"
    done
```

Format rules: [the file format](https://llms-full-txt.ru/en/specification/format/), [sections](https://llms-full-txt.ru/en/specification/sections/).

## Checking llms-full.txt

- It answers `200` with a text content type.
- It holds the text of pages, not a list of links.
- Its size is known. An agent will not read a file of tens of megabytes in one go.
- It is built from the same content as the site and refreshed together with it.

```bash
curl -sI https://example.com/llms-full.txt | grep -i content-length
```

When this file is needed and when it is not: [What is llms-full.txt](https://llms-full-txt.ru/en/guides/llms-full-txt/).

## Checking Markdown versions and relations

```bash
curl -sI -H 'Accept: text/markdown' https://example.com/docs/install/
curl -sI https://example.com/docs/install/ | grep -i '^link'
curl -s https://example.com/docs/install/ | grep -iE 'rel="(alternate|describedby)"'
```

- A request with `Accept: text/markdown`, or the address ending in `.md`, returns text, not HTML.
- The address in `rel="alternate"` answers `200`.
- The address in `rel="describedby"` points to an existing `llms.txt`.

More: [Markdown versions of pages](https://llms-full-txt.ru/en/guides/markdown-pages/), [discovery](https://llms-full-txt.ru/en/specification/discovery/).

## Checking structured data

| Tool | What it checks | Address |
|---|---|---|
| [Rich Results Test](https://search.google.com/test/rich-results) | which rich results Google can show, errors and warnings | `search.google.com/test/rich-results` |
| [Schema Markup Validator](https://validator.schema.org/) | conformance to the Schema.org vocabulary, independent of Google | `validator.schema.org` |
| [Yandex structured data validator](https://webmaster.yandex.ru/site/tools/microtest/) | how Yandex sees the markup | `webmaster.yandex.ru/site/tools/microtest/` |
| Search Console | markup errors across the whole site over time | enhancement reports |

1. The `application/ld+json` block is in the server response, not added later by scripts.
2. The JSON is valid: the tool opens it without a parse error.
3. The type matches the page: a product is `Product`, an article is `BlogPosting` or `Article`.
4. Required properties are filled. For a product: name, offer, price, currency, availability.
5. Values match the visible page: price, availability, author, date.
6. Each entity is described once, with no duplicate from the theme.
7. Breadcrumbs in the markup match the breadcrumbs on the page.
8. Pages of different kinds were checked: a product with variants, an out-of-stock product, an article, the home page.

```bash
curl -s https://example.com/product/kettle/ \
  | grep -c 'application/ld+json'
```

Zero means the server response has no markup. More than one is a reason to look for duplicated entities.

Which types are needed and what goes in them: [Structured data and JSON-LD](https://llms-full-txt.ru/en/guides/json-ld/).

## Typical mistakes

- /llms.txt answers 200, but it is HTML
- The site serves the home page or an error page for any unknown address. Lighthouse and an agent see a file that does not exist.
- The content type is text/html and the first line starts with a tag.

- Links in the file are relative or point to old addresses
- An agent reads the file outside the site and cannot build the address. Some links answer 404.
- The list has paths without a domain, or addresses that redirect.

- The price in the markup differs from the price on the page
- The theme prints markup from one field while a discount module shows the on-screen price from another.
- The validator shows no errors: it checks form, not truth.

- The file was created once and never refreshed
- The generator was run by hand. The site changed, the map did not.
- The file lists pages that are gone and misses new sections.

## What a check does not show

A validator confirms form. It does not know whether the price is right or whether anyone reads the file.

Whether the file is actually read shows only in server logs: [How to measure AI visibility](https://llms-full-txt.ru/en/guides/ai-visibility/).

Lighthouse checks `llms.txt` too, but only the fact of a response: [Lighthouse and llms.txt](https://llms-full-txt.ru/en/guides/lighthouse/).