Getting Started

Create your first llms.txt in 5 minutes — step-by-step guide with code examples, file template, validation and next steps

What is llms.txt?

llms.txt is a Markdown file placed at /llms.txt on your website. It’s a community-driven convention (not a formal W3C/IETF standard), proposed by Jeremy Howard (Answer.AI / fast.ai) in September 2024.

The file helps language models (Claude, ChatGPT, Gemini) understand your site structure and find the right information. Used by Anthropic, Cloudflare, Stripe, and hundreds of other sites.

The Problem

LLM context windows are limited (typically 100-200K tokens)
HTML pages contain lots of “noise” (navigation, ads, scripts)
LLMs cannot process an entire website at once

The Solution

The llms.txt file provides:

A brief project description
A structured list of important pages
Descriptions of each resource for LLMs

Creating llms.txt Manually

Step 1: Create the File

Create a llms.txt file in your site’s root:

# My Project

> Brief project description — what it does and who it's for.

Additional information about the project to help LLMs understand the context.

## Documentation

- [Quick Start](https://example.com/docs/quickstart): Step-by-step guide for new users
- [Installation](https://example.com/docs/installation): Requirements and installation instructions
- [API Reference](https://example.com/docs/api): Complete description of all endpoints

## Guides

- [Authentication](https://example.com/guides/auth): How to set up authorization
- [Deploy](https://example.com/guides/deploy): Production deployment

## Optional

- [Changelog](https://example.com/changelog): Version history
- [Contributing](https://example.com/contributing): How to contribute to the project

Step 2: Deploy to Your Site

The file should be accessible at:

https://yoursite.com/llms.txt

Step 3: Verify

Open the URL in a browser and make sure the file displays correctly.

Automatic Generation

For large sites, use automation tools:

# Generate from sitemap.xml
npx llmstxt gen https://example.com/sitemap.xml > llms.txt

# Generate with full content
npx llmstxt gen-full https://example.com/sitemap.xml > llms-full.txt

Integrations

Astro + Starlight

Use the starlight-llms-txt plugin:

pnpm add starlight-llms-txt

import starlightLlmsTxt from 'starlight-llms-txt'

export default defineConfig({
  integrations: [
    starlight({
      plugins: [starlightLlmsTxt()],
    }),
  ],
})

VitePress

npm install vitepress-plugin-llms

Docusaurus

npm install docusaurus-plugin-llms-txt

Next Steps

Standard Status — adoption, debates, and outlook
File Format — detailed specification
Sections — required and optional sections
Best Practices — recommendations for effective llms.txt
AI Crawlers — managing AI bot access
Generators — full tool catalog