Mintlify test: with llms.txt an agent hits fewer missing pages
Mintlify compared four documentation formats on 20 sites and 2,400 runs - what changed, what stayed the same and why the source has to be treated as an interested party
On July 17, 2026, the documentation platform Mintlify published a comparison: how agents cope with questions about documentation when they get it in different formats.
The source is an interested party. Mintlify creates llms.txt and Markdown versions for its customers, and the test ran on sites of its own platform.
What was compared
Four variants of the same documentation:
- pages in HTML;
- pages in Markdown;
- Markdown with a link to
llms.txt; - Markdown with the content of
llms.txtinside the page.
Agents running OpenAI and Anthropic models were asked five questions on each of 20 sites, three times in each format. 2,400 runs in total.
What came out
| Format | Requests to missing addresses per task |
|---|---|
| HTML | 2.23 |
| Markdown | 1.42 |
Markdown and llms.txt | 0.11 |
Answer accuracy barely changed: in every format it stayed above 90%.
How to read it
The file did not make answers more correct. It cut the wandering: with a map, an agent stops guessing addresses and follows links.
This matches how the second edition of the proposal describes the file: an agent scans llms.txt and follows the links it needs.
What the test does not show
- That bots of AI search systems behave the same way. The test covered agents that were given a task and a site.
- That the result carries over to shops and blogs. All 20 sites are documentation on one platform.
- That a site will be mentioned in answers more often.
What a site owner should do
If you run documentation, the benefit is measurable: fewer wasted agent requests. How to serve Markdown and connect it with the file: Markdown versions of pages.