Site crawler for internal links and technical SEO
Crawl your whole site to see how pages connect, where internal links are missing and which pages compete for the same searches. Available on Pro and Agency plans.
What it checks
Everything the site crawl looks at
Whole-site crawl
Crawls up to 100 pages on Pro and 500 on Agency, honouring your robots.txt rules.
Indexability, canonical and redirect audit
Finds noindexed pages, canonical conflicts and redirects across the site, not just on one page.
Sitemap versus crawl reconciliation
Compares your XML sitemap with what the crawl found to reveal orphaned pages and stale sitemap entries.
Internal link opportunities
Suggests links between related pages that are not yet connected, to help visitors and search engines.
Interactive internal link map
Visualises how your pages link to each other so weakly linked areas stand out.
Page types and topic clusters
Classifies pages by type and groups them into topic clusters to show how your content is organised.
Funnel coverage
Shows how your content spreads across top, middle and bottom of funnel (TOFU, MOFU, BOFU).
Keyword cannibalisation
Flags pages that appear to compete for the same search, which can split rankings between them.
How it works
Three steps, about a minute.
-
1
Start a crawl
Enter the site address on the crawl page and start the crawl.
-
2
We follow your links
Pages are discovered through internal links and your sitemap, respecting robots.txt.
-
3
Explore the results
Use the link map, clusters and issue lists to plan fixes and new internal links.
What a site crawler reveals that a single-page audit cannot
A single-page audit tells you whether one page is set up well. A site crawler follows your internal links across the whole site, which reveals problems that only appear in aggregate. Pages may be listed in your sitemap but not linked from anywhere, or linked internally but missing from the sitemap. Canonical tags may point to redirected URLs, and important pages may be several clicks deep with few links pointing to them. The MakeItFlow site crawl audits up to 100 pages on Pro and 500 on Agency, honours robots.txt and reconciles your sitemap with what it actually finds, so you get a clear list of indexability, canonical and redirect issues across the site.
Using crawl data for internal linking and content strategy
Beyond technical checks, the crawl helps you understand how your content fits together. Each page is classified by type and grouped into topic clusters, and the interactive link map shows which clusters are well connected and which are isolated. Funnel coverage shows whether you have enough content for people researching a problem, comparing options and ready to buy. The internal link suggestions point out related pages that should link to each other but do not. Keyword cannibalisation detection highlights pages that seem to target the same search, which can split ranking signals; the usual fixes are merging them, differentiating their focus or making one clearly the primary page.
Why it matters
See the whole picture
Site-wide problems such as orphaned pages and canonical conflicts only show up when every page is checked.
Stronger internal linking
Concrete link suggestions help pass authority to important pages and guide visitors to related content.
Fix competing pages
Cannibalisation detection shows where to merge, differentiate or re-target pages.
Who it is for
Growing businesses
Keep a content-heavy site organised as it grows beyond a handful of pages.
Agencies
Deliver site-wide technical SEO audits and content strategy recommendations for clients.
SEO specialists and content teams
Plan internal linking, topic clusters and content gaps from real crawl data.
Try the free tools
Learn more
The site crawl is available on Pro, which crawls up to 100 pages, and Agency, which crawls up to 500. See pricing for details.
Yes. Pages disallowed in your robots.txt are not crawled.
It is when two or more of your pages target the same search, so they compete with each other instead of one strong page ranking well.
The crawler fetches a handful of pages at a time and honours robots.txt, so it should not noticeably affect a normally hosted site.
Free website audit
See how your site scores, free
Speed, SEO, DNS, email (SPF/DKIM/DMARC) and security in one scan, with a prioritised fix list.