AI search readiness check for your website
AI answers in Google, Copilot, ChatGPT and Perplexity can only cite pages their crawlers can reach and read. See what your site allows, what it blocks and what those systems can see.
What it checks
Everything the ai search readiness looks at
Googlebot and Bingbot access
Checks robots.txt allows the crawlers whose index feeds Google AI Overviews and Microsoft Copilot.
AI search crawlers
Checks access for OAI-SearchBot (ChatGPT search), Claude-SearchBot and PerplexityBot, which fetch pages to cite in answers.
AI training crawlers
Reports your rules for GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, Meta-ExternalAgent and CCBot separately, as a policy choice that is never penalised.
Snippet controls
Detects nosnippet and max-snippet directives that can limit how much of your content is shown or quoted.
Text without JavaScript
Measures how much text is present in the raw HTML, since many crawlers do not run JavaScript.
Organization or LocalBusiness schema
Checks for structured data that helps search and AI systems identify your business accurately.
llms.txt
Notes whether an llms.txt file exists, for information only; it is not a ranking factor and Google says it does not use it.
How it works
Three steps, about a minute.
-
1
Enter your URL
Run a normal audit; AI search readiness is included in the report.
-
2
We check crawler access
Your robots.txt, meta directives, HTML content and schema are reviewed from an AI crawler's point of view.
-
3
Decide and adjust
Allow the search crawlers you want citing you, and set training crawler rules according to your own policy.
What AI search readiness means
AI search readiness is about whether AI-powered answers can find, read and cite your pages. Google AI Overviews draws on Google's own search index, and Microsoft Copilot draws on Bing, so the first requirement is that Googlebot and Bingbot can crawl your site. Separate AI search crawlers, such as OAI-SearchBot for ChatGPT search, Claude-SearchBot and PerplexityBot, fetch pages to cite in their answers, and they are controlled through robots.txt too. Content also needs to be readable: many crawlers do not execute JavaScript, so text that only appears after scripts run may be invisible to them. Clear Organization or LocalBusiness schema helps these systems describe your business accurately.
AI search crawlers versus AI training crawlers
Not every AI crawler serves the same purpose. Search crawlers fetch pages to answer a user's question and usually link back to the source, which can send you visitors. Training crawlers, such as GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, Meta-ExternalAgent and CCBot, collect content to train or improve models. Many sites choose to allow search crawlers while blocking training crawlers, and that is a reasonable policy. Our report lists the two groups separately and never penalises you for blocking training. AI search readiness is shown on every report but is not counted in your overall score, because good practice in this area is still settling and the right choices depend on your business.
Why it matters
Search and training kept separate
Blocking training crawlers is a legitimate choice, so we report it separately and never mark you down for it.
Find accidental blocks
A broad robots.txt rule can block search crawlers you actually want; the check makes these visible.
Grounded in what is known
We focus on access, readable content and schema rather than speculative tactics, and say plainly that llms.txt is informational.
Who it is for
Small business owners
Make sure AI assistants can find and describe your business when customers ask about services like yours.
Agencies
Answer client questions about AI search with a concrete, evidence-based report.
Publishers and content teams
Review which AI crawlers you allow and set a deliberate policy for search versus training.
Try the free tools
Learn more
-
Block or Allow AI Crawlers? GPTBot, ClaudeBot and MoreSEO · 10 min read
-
How ChatGPT Search Chooses Sources: The Mechanics ExplainedSEO · 10 min read
-
How to Get Into Google AI Overviews and AI ModeSEO · 10 min read
-
AI Search Visibility Tools: A Sober Guide for Small Business SEOSEO News · 6 min read
-
Google AI Mode and AI Overviews in the UK: The Impact on ClicksSEO News · 6 min read
-
Robots.txt Guide: Syntax, Examples and Common MistakesSEO · 7 min read
-
Bing Webmaster Tools AI Reports: Why UK Websites Should CareSEO News · 5 min read
Not necessarily. GPTBot is used for training, while OAI-SearchBot fetches pages for ChatGPT search. You can block one and allow the other.
It is optional. We report whether one exists, but it is not a ranking factor and Google has said it does not use it.
No. AI search readiness appears on every report but is not counted in the overall A-F score.
There is no separate opt-in. Pages need to be crawlable and indexable by Googlebot and eligible for snippets, so good technical SEO is the foundation.
Free website audit
See how your site scores, free
Speed, SEO, DNS, email (SPF/DKIM/DMARC) and security in one scan, with a prioritised fix list.