332 practical guides to website speed, SEO, email deliverability, DNS, security and automation, with fixes you can copy and paste.
AI crawlers overloading your server? Find the bot in your logs, block it at the server for instant relief, then use robots.txt and 429 responses long term.
Google and Microsoft said on October 30, 2026 they will honor a shared AI training robots.txt signal, __usage_ai-training. What it does and how to prepare.
We checked the 10,000 most popular websites for AI crawler blocks, llms.txt, HTTPS, security headers, SPF and DMARC. 22.4% of sites with a robots.txt block at least one AI bot.
Cloudflare and OpenAI shared results of a pilot that fed network signals to ChatGPT search on October 15, 2026, plus new checks on bot behavior. What to do.
Cloudflare replaced Block AI Bots with Search, Training and Agent controls on 15 September 2026. What sites should check to stay visible in search.
An AI startup website checklist for UK founders: AI crawler controls, llms.txt, trust and security pages, headers, and claims that satisfy the ASA and CMA.
A practical way to test robots.txt changes: how precedence and wildcards really match, per-crawler groups including AI bots, and which tools to trust.
Should you block AI crawlers like GPTBot, ClaudeBot and PerplexityBot? Learn which bots train models, which power AI search, and what each block costs you.
llms.txt is a proposed Markdown file that summarises your site for AI tools. Google Search ignores it, so most businesses do not need one. See when it helps.
Popular topics
Free website audit
Speed, SEO, DNS, email (SPF/DKIM/DMARC) and security in one scan, with a prioritised fix list.
Make It Flow assistant
Tools, guides and AI automation
Hi! I can find the right free tool, explain an audit result, or tell you how Make It Flow can automate any part of your business. What do you need?
Book a free 15-minute call
Pick a time that suits you. Times are shown in your time zone ().
A Make It Flow agent will confirm the call by email within 24 hours.
AI answers can be wrong. Chats are stored so the team can follow up. Privacy