llms.txt Checker & Generator

An llms.txt checker tells you whether a website publishes an llms.txt file and whether that file follows the format proposed at llmstxt.org. Enter a site and this tool looks for /llms.txt at the root, confirms it is a real text file rather than an HTML page, checks the required H1 title, the summary blockquote and the H2 link sections, tests a sample of the links inside it, and estimates how long it is in tokens. If the site has no file yet, it builds a starter draft from the home page title, meta description and sitemap, so you have something concrete to edit instead of a blank page. It is honest about what the file can and cannot do: llms.txt is an optional proposal, not a ranking factor.

How to use the llms.txt checker

1. 1 Enter the websiteType the domain or any page address. The checker always looks for the file at the root of that host, such as https://example.com/llms.txt.
2. 2 Run the checkWe fetch /llms.txt through our safety checks, note the status code and content type, and make sure the response is text rather than an HTML page the site sends for unknown addresses.
3. 3 Read the format resultsSee whether the file starts with an H1, has a summary, groups links under H2 headings in "- [name](url): notes" form, and uses full URLs.
4. 4 Fix broken links and gapsUp to 10 links are tested. Update or remove any that return errors, and add short notes to links that have none.
5. 5 No file? Use the draftCopy the starter draft, replace the placeholder notes with real descriptions, delete anything unhelpful and upload it to your site root.

What llms.txt is

llms.txt is a plain Markdown file placed at the root of a website, at /llms.txt, that gives large language models and AI agents a short, curated guide to the site. It was proposed in September 2024 by Jeremy Howard of Answer.AI and is described at llmstxt.org. The idea is simple: web pages are full of navigation, scripts and layout that make it hard for a model working with a limited context window to find the useful content, so the site owner writes a compact summary with links to the pages that matter most.

Think of it as a hand-written table of contents for machines. Where a sitemap lists every URL for crawlers and robots.txt says what bots may fetch, llms.txt says "if you only read a few pages, read these, and here is what each one covers". It is most common on developer documentation, where a model answering a coding question benefits from a direct path to the right reference page, but any site can publish one.

It is important to be clear about its status. llms.txt is a community proposal, not a web standard, and it is not part of robots.txt or the sitemap protocol. Google representatives have said publicly that Google Search does not use it. Some AI tools and coding assistants can read it when a user points them at a site, but no major AI search product has documented that it relies on the file. Publishing one is cheap and harmless; expecting it to change rankings or citations is not supported by evidence.

The llms.txt format, section by section

The proposal defines a small, predictable structure so that both people and programs can read the file. Only the first element is required. Everything else is recommended, and the order matters: title, summary, optional detail, then sections of links.

Each link line follows the same pattern: a Markdown list item, the page name in square brackets, the URL in round brackets, and optionally a colon followed by a few words about the page. Keeping to that pattern is what lets a program pull out the links reliably, which is why the checker flags list items inside link sections that do not match it.

• H1 title (required): one line starting with "# " that names the site or project.
• Summary blockquote: a line starting with "> " with a one- or two-sentence description of what the site offers.
• Detail paragraphs (optional): plain Markdown with any context a model should know, without headings.
• H2 link sections: headings starting with "## " such as Docs, Services or Guides, each followed by a list of "- [name](url): notes" lines.
• An "Optional" section: an H2 named Optional for links that can be skipped when a shorter context is needed.

What this checker tests, and why

First it confirms the file exists and is really a text file. A surprisingly common failure is a site that answers every unknown address with its home page and a 200 status, so /llms.txt appears to load but actually returns HTML. The checker detects that and reports the file as missing. It also notes the content type, because a file served as text/html instead of text/plain or text/markdown may be treated as a web page by tools that read it, and it flags redirects to a different path.

Next it parses the Markdown. It checks that the first non-empty line is an H1, warns if there are several H1 lines, and looks for the summary blockquote between the title and the first section. It counts the H2 sections and the links inside them, lists any list items that are not in link format, and warns about relative URLs such as /pricing, since a model reading the file on its own has no page to resolve them against. If none of the links has a description, it suggests adding short notes.

Finally it tests up to 10 absolute links with lightweight requests and reports any that return errors or do not respond, reports the size in characters with a rough token estimate (about four characters per token, which varies by model and language), and checks whether the site also publishes llms-full.txt. That second file is a convention some documentation platforms use to put the full text of their docs in one place; it is not part of the original proposal, so the checker reports it for information only.

How to write a useful llms.txt

Start with the questions people actually ask about your business or product, then pick the pages that answer them best: what you offer, pricing, how it works, key guides or documentation, contact and policies. A small business site might need fewer than twenty links. A documentation site might need more, but it should still be a curated list, not a copy of the sitemap. Put secondary material such as the blog archive, press pages or legal small print under the Optional heading.

Write the summary for a reader who has never heard of you. Say what you do, who it is for and where you operate, in plain language, without slogans. Then give each link a short, specific note: "plans, limits and what each tier includes" is far more useful than "pricing page". Use the same names you use on the site so the file and the pages agree, and keep the file updated when pages move, because a map full of broken links is worse than no map.

Some sites also publish a Markdown version of key pages, for example at the same address with .md added, and link to those from llms.txt so a model gets clean text without layout. That is part of the original proposal but optional, and it only makes sense if you can keep the Markdown copies in sync with the real pages.

How to publish llms.txt on common platforms

On any host where you can upload files, save the file as llms.txt in UTF-8 and place it in the web root, the same folder that holds robots.txt, so it loads at https://yourdomain.com/llms.txt. Most servers will serve a .txt file as text/plain automatically. Open the address in a browser afterwards: you should see the raw Markdown, not your home page.

On WordPress, upload it with your host's file manager or SFTP to the folder that contains wp-config.php, or use an SEO plugin that offers an llms.txt option; some, including Yoast SEO, can generate one for you. On hosted builders such as Shopify, Wix or Squarespace, you may not be able to place arbitrary files at the root, so check the platform's current documentation or app store before you plan around it.

If your site is behind a CDN or security service, make sure /llms.txt is not cached as an HTML error page and is not blocked for the AI crawlers you want to reach it. Run this checker again after publishing to confirm the file loads, parses cleanly and that its links work.

llms.txt, robots.txt and what actually affects AI visibility

llms.txt does not grant or deny access to anything. If you want to allow or block AI crawlers, that is done in robots.txt, and at the server or CDN level for bots that ignore it. A site that blocks OAI-SearchBot or PerplexityBot in robots.txt will not be fetched by them regardless of what llms.txt says.

The factors with clearer evidence behind them are the same ones that help ordinary search: pages that crawlers are allowed to fetch, content present in the HTML without needing JavaScript, clear headings and direct answers, accurate structured data, and a site people trust and link to. Use our AI Crawler Checker to review robots.txt rules for AI bots and the AI Content Visibility Checker to see how a single page reads to them, then treat llms.txt as a tidy, low-cost extra on top.

Examples you can copy

# Example Plumbing Co.

> Licensed plumbers serving Austin, Texas: emergency repairs, water heaters and drain cleaning for homes and small businesses.

## Services

- [Emergency plumbing](https://example.com/emergency): 24/7 call-outs, response times and pricing
- [Water heaters](https://example.com/water-heaters): repair, replacement and tankless options

## Optional

- [Blog](https://example.com/blog): maintenance tips and seasonal advice

# Acme API

> REST API for sending invoices and collecting card payments.

All endpoints use JSON and require an API key in the Authorization header.

## Docs

- [Quickstart](https://docs.example.com/quickstart.md): create a key and send a first invoice
- [Authentication](https://docs.example.com/auth.md): keys, scopes and rotation
- [Webhooks](https://docs.example.com/webhooks.md): events, retries and signature checks

## Optional

- [Changelog](https://docs.example.com/changelog.md): release history

Example Plumbing Co.            <- no '# ' so there is no H1

## Services

- Emergency plumbing           <- not in [name](url) form
- [Pricing](/pricing)          <- relative URL

Common mistakes to avoid

Frequently asked questions