What Is llms.txt and Do I Need One for My Website?

llms.txt is a proposed standard: a Markdown file at yoursite.com/llms.txt that gives AI tools a short summary of your site and links to its most useful pages. It is optional, not an official standard, and Google Search ignores it, so it will not help rankings or AI Overviews. Most businesses do not need one, though it can help documentation sites.

Below we explain where the idea came from, what the file looks like, what Google and others have actually said, when it is worth the ten minutes, and what matters more for AI visibility.

Key takeaways

llms.txt is a proposed Markdown file that summarises your site for AI tools, and it is not an official standard.
Google Search ignores llms.txt, so it will neither help nor harm your rankings or visibility in AI Overviews.
Most businesses do not need one, though software and documentation sites may benefit because some developer tools read it.
Crawler access, indexing, server-rendered text and clear original content matter far more for AI visibility than llms.txt.

Where did llms.txt come from?

Jeremy Howard of Answer.AI proposed llms.txt in September 2024. The reasoning was sensible: web pages are full of navigation, scripts and adverts, and language models have limited context windows, so a clean, curated summary in Markdown would help an AI tool understand a site quickly. The proposal also suggests offering Markdown versions of key pages, for example page.html.md alongside page.html.

It is a community proposal published at llmstxt.org, not a standard from the W3C or IETF, and no major search engine has committed to using it.

What does an llms.txt file look like?

The specification asks for a specific Markdown structure:

An H1 with the site or project name (the only required part).
A blockquote with a one-line summary.
Optional paragraphs of detail.
H2 sections containing lists of links, each written as [name](url): optional note.
An optional section headed Optional for links an AI tool can skip if it is short on context.

A small business example:

# Harbour Joinery

> Bespoke kitchens, wardrobes and staircases made in our Bristol workshop and fitted across the South West.

We are a family business founded in 2009, members of the relevant trade association, with a 10-year guarantee on all fitted furniture.

## Services

- [Bespoke kitchens](https://harbourjoinery.co.uk/kitchens): design, build and fitting, typical lead time 8–10 weeks
- [Fitted wardrobes](https://harbourjoinery.co.uk/wardrobes): alcove, loft and walk-in designs
- [Staircases](https://harbourjoinery.co.uk/staircases): oak and painted softwood

## Company

- [About us](https://harbourjoinery.co.uk/about)
- [Areas we cover](https://harbourjoinery.co.uk/areas)
- [Contact and quotes](https://harbourjoinery.co.uk/contact)

## Optional

- [Project gallery](https://harbourjoinery.co.uk/projects)

Save it as plain text with UTF-8 encoding at the root of your site, so it is served at https://yourdomain.com/llms.txt.

Does Google use llms.txt?

No. Google's Search Central guide to generative AI features says you do not need to create machine-readable files, AI text files or Markdown versions to appear in Google Search, because Google Search does not use them. It adds that creating them will neither harm nor help your visibility or rankings. Google's John Mueller has publicly compared llms.txt to the old keywords meta tag, and Gary Illyes has said Google is not pursuing it.

There is one wrinkle. Chrome's Lighthouse added an "Agentic Browsing" audit category in 2026 that includes an llms.txt check. It describes the file as an "emerging convention" for AI agents. Crucially:

If your site returns a 404 for /llms.txt, the audit is marked not applicable, not failed.
It only flags a problem if requesting the file causes a server error.
The category is separate from Lighthouse's SEO score and does not affect rankings.

So a missing llms.txt is fine. A misconfigured server that throws a 500 error when the file is requested is worth fixing anyway.

Do ChatGPT, Claude or Perplexity use llms.txt?

There is no public commitment from the major AI assistants to read llms.txt when answering questions, and Google's John Mueller has pointed out that AI services did not appear to be requesting the file at all. Check your own server logs if you are curious. Where it does get used is in developer tools: some coding assistants and documentation platforms read llms.txt (and the fuller llms-full.txt some sites publish) to load API docs into context. That is why software companies with large documentation sites are the main adopters.

Do I need an llms.txt file?

Your situation	Recommendation
Small business brochure site	Not needed. Spend the time on clear service pages
Online shop	Not needed for search; product data and structured data matter more
Software, API or technical documentation	Worth considering, as developer tools may use it
Agency managing client sites	Optional; do not sell it as an SEO or AI visibility fix
You already have one	Fine to keep, as long as it is accurate

If you decide to add one, keep it short and accurate, and update it when your services change. A stale llms.txt with old prices or discontinued services is worse than none.

What matters more than llms.txt for AI visibility?

The things that actually decide whether AI tools can find and cite your site are the same fundamentals that matter for search:

Crawler access. Make sure robots.txt and your firewall are not blocking AI search crawlers such as OAI-SearchBot. Our AI crawler checker tests this quickly. If you do publish an llms.txt file, our llms.txt checker validates its structure and links, and builds a starter draft from your sitemap if you have none.
Indexing. Google requires a page to be indexed and snippet-eligible to appear in AI Overviews. If pages are not indexed, read crawled, currently not indexed.
Server-rendered text. Key facts should be in the HTML, not only injected by JavaScript.
Snippet controls. nosnippet or very low max-snippet values can exclude pages from AI features.
Clear, original content that answers real questions, and a genuine reputation on other sites. Our guide on getting your business mentioned in ChatGPT covers this in detail.

Is llms.txt the same as robots.txt?

No. robots.txt is a long-established standard (RFC 9309) that tells crawlers what they may and may not fetch, and well-behaved crawlers obey it. llms.txt grants or blocks nothing; it is a reading list. If you want to control AI crawlers, use robots.txt. The MakeItFlow robots.txt generator can create rules for specific bots, and our robots.txt guide explains the syntax.

Frequently asked questions

Can llms.txt hurt my SEO?

Google says it will neither help nor harm rankings, because Google Search ignores it. The only risk is publishing inaccurate information that a tool might repeat.

What is llms-full.txt?

It is a related convention where a site publishes the full text of its documentation as one Markdown file, rather than a list of links. It is mainly used by software documentation platforms for coding assistants.

Should agencies charge clients for llms.txt?

Be careful. Selling it as a ranking or AI-citation fix is not supported by the evidence. If you add one, present it as an optional, low-cost extra.

Do I need Markdown versions of every page?

No. Google explicitly says Markdown copies are not needed for Search. Duplicating your site in another format also creates more content to maintain.

Where should the file go?

At the root of your domain, served as plain text, for example https://example.com/llms.txt. Subfolder locations are not part of the main proposal.

What to do next

If you want a quick view of how AI-ready your site really is, check crawler access, snippet rules and rendered content before worrying about llms.txt. MakeItFlow includes an AI search readiness check that covers robots rules for AI crawlers, snippets, server-rendered text, schema and whether an llms.txt file is present, as part of a free website audit.

Cloudflare's New AI Crawler Controls: What Websites Should Check: Cloudflare replaced Block AI Bots with Search, Training and Agent controls on 15 September 2026. What sites should check to stay visible in search.