Indexability Checker (noindex, canonical, robots)

An indexability checker answers one question: is anything technical stopping Google from indexing this page? Our free tool, which also works as a noindex checker, follows the redirect chain, reads the status code, the robots meta tag and X-Robots-Tag header, checks the canonical and whether its target loads, finds the robots.txt rule that applies to Googlebot, looks for the URL in your sitemap and counts hreflang tags. You get a clear verdict, Indexable or Not indexable because of a named reason, with the fix for each problem.

How to use the Indexability checker

1. 1 Sign in and open the checkerLog in to your free account and open the Indexability Checker from the tools list.
2. 2 Paste the exact URLEnter the full page address as it appears in search results or your sitemap, including the path and any trailing slash.
3. 3 Read the verdict firstThe top line tells you whether the URL is indexable, indexable with a caveat, or not indexable and why.
4. 4 Fix the critical itemsWork through anything marked Critical, such as noindex, a robots.txt block or an error status, before warnings.
5. 5 Ask Google to recrawlOnce fixed, use URL Inspection in Google Search Console to request indexing for important pages.

What indexable means

Before a page can appear in Google, three things must happen: Google must be allowed to crawl it, the page must load successfully, and nothing on the page or in its headers may tell Google to leave it out. A page that passes those tests is indexable. That does not guarantee it will be indexed, because Google also decides whether the page is useful and not a duplicate, but a page that fails them will not be indexed however good it is.

This checker tests the technical side as Googlebot would see it. It requests the page without following redirects automatically, so it can show every hop; reads the final response; and then checks every signal that can block or redirect indexing. The verdict lists every blocking reason it finds, not just the first one, because pages often have more than one problem at once.

• Not indexable: the URL redirects, returns an error, is blocked by robots.txt, or carries noindex.
• Indexable, with a caveat: nothing blocks it, but a canonical points elsewhere or robots.txt is failing, so Google may index another URL or pause crawling.
• Indexable: no technical obstacle found.

Status codes and redirect chains

Only pages that return HTTP 200 can be indexed. A 404 or 410 tells Google the page does not exist, and it will be dropped. A 5xx server error repeated over several days has the same effect. A 3xx redirect means the URL itself is not indexed: Google follows it and considers the destination instead.

The checker follows redirects one hop at a time, up to ten, and shows the full chain with each status code. Permanent redirects (301 and 308) are the right choice for moved pages. Temporary redirects (302 and 307) can keep the old URL in the index longer. Long chains waste crawl time and slow visitors down, so point every old URL straight at its final destination. A redirect loop never reaches a page at all and is always a fault.

noindex in the robots meta tag and X-Robots-Tag header

A noindex directive tells search engines not to show a page in results. It can be set in the page with a robots meta tag, or by the server with an X-Robots-Tag HTTP header. Both work the same way, but the header is invisible in the page source, which is why it is so often missed. The checker reads both, including directives aimed specifically at Googlebot, such as a googlebot meta tag or an X-Robots-Tag value starting googlebot:.

The value none means noindex and nofollow together, so it also blocks indexing. The unavailable_after directive removes a page after a given date, and the checker treats a date in the past as noindex. A nofollow on its own does not stop the page being indexed, but it does stop Google following the links on it, which the checker reports for information.

Common causes of an accidental noindex are the WordPress setting Discourage search engines from indexing this site, an SEO plugin option applied to a whole post type, a staging site setting carried over to live, or a header added by a hosting control panel, CDN rule or security plugin.

Canonical tags: self, other or missing

A canonical tag names the preferred URL for a page. When it points to the page itself, it confirms that this is the version to index and folds tracking parameters and minor variations into it. When it points to a different URL, it tells Google this page is a duplicate and the other address should be indexed instead. Google treats the canonical as a strong hint rather than a command, but it usually follows it.

The checker reads canonicals from the HTML and from Link HTTP headers. It reports a self-referencing canonical as a pass, a missing one as a warning, and a canonical to another URL as a caveat in the verdict. It also flags relative canonicals, which Google accepts but which can go wrong on staging copies, and conflicting canonicals, where two tags point to different places and Google may ignore both. When the canonical points elsewhere, the checker requests that URL to make sure it loads and does not redirect, because a canonical to a broken or redirecting page sends mixed signals.

robots.txt rules and the sitemap

robots.txt controls crawling, not indexing. If it blocks a URL, Google cannot fetch the page, so it cannot read the content or see a noindex tag. Google may still list the bare URL if other sites link to it, which is why Search Console has a status called Indexed, though blocked by robots.txt. The checker finds the exact rule that decides the URL for Googlebot, using the same matching rules as Google: Googlebot's own group if there is one, otherwise the * group, with the longest matching path winning.

If robots.txt returns a 4xx error, Google treats the site as having no restrictions. If it returns a 5xx error or times out, Google stops crawling the site until it loads again, so the checker treats that as a caveat.

The checker also looks for the URL in the sitemaps listed in robots.txt, or in the usual locations if none are listed. It reads a limited sample to stay fast: the first few sitemap files and up to 64 KB of each. It tells you whether the exact URL is listed, a near variant (different protocol, www or trailing slash) is listed, or an old redirecting address is listed instead. Being missing from the sitemap does not block indexing, but important pages should be in it.

Indexable but not indexed? What to check next

If the checker says a page is indexable but it still does not appear in Google, the cause is no longer technical. Open URL Inspection in Search Console to see Google's own status for the URL. Crawled, currently not indexed means Google fetched the page and decided not to index it for now, usually because it sees little unique value. Discovered, currently not indexed means Google knows about the URL but has not crawled it yet, often on new or large sites.

Improve the page so it is clearly the best answer for its topic, link to it from relevant pages on your own site, make sure it is in the sitemap, and avoid publishing many near-identical pages. Duplicate, Google chose different canonical than user means Google picked another URL as the main version; compare the two pages and make them clearly distinct, or redirect one to the other.

For pages in several languages, also check the hreflang annotations with our free hreflang checker, because errors there can make Google show the wrong country version rather than drop the page.

Examples you can copy

<link rel="canonical" href="https://www.example.co.uk/services/boiler-repair/">
<meta name="robots" content="index, follow">

<meta name="robots" content="noindex, follow">

<Files "price-list.pdf">
  Header set X-Robots-Tag "noindex"
</Files>

User-agent: *
Disallow: /search
Sitemap: https://www.example.co.uk/sitemap.xml

Common mistakes to avoid

Frequently asked questions