To automate broken link checks, schedule a link checker to run weekly against your live site, have it report only failures, and send them somewhere you'll act on, such as a GitHub issue or an email. lychee and linkinator are free, open source and work well in a GitHub Actions cron job.
A one-off check with our free broken link checker tells you what's broken today. Links keep breaking after that. Suppliers restructure their sites, someone deletes a page without adding a redirect, a PDF gets renamed. This guide covers the automation side: what to schedule, how to configure it without hammering other people's servers, and how to fix hundreds of links at once. For the basics of why 404s and redirect chains matter, read how to find and fix broken links and redirect chains first.
Key takeaways
- Broken links pile up between manual checks, so a weekly scheduled scan finds them before visitors and crawlers do.
- lychee checks the links on pages you give it but does not crawl, so feed it your sitemap URLs, while linkinator can recurse through a live site.
- GitHub Actions scheduled workflows run in UTC, can be delayed at busy times and are disabled in public repositories after 60 days without activity.
- Check external links slowly with low concurrency and caching, and treat 403, 429 and 999 responses as needing a manual look rather than as broken.
What to automate (and what not to)
Split your links into three groups, because each needs different handling:
| Link type | Check how often | Why |
|---|---|---|
| Internal links | Weekly, and on every deploy if you can | You control them, so every failure is a bug you can fix |
| Redirected internal links | Weekly | Not broken, but each hop wastes crawl effort and slows visitors |
| External links | Weekly or monthly, slowly | Other people's sites; you can only update or remove your link |
Don't automate fixes. A script that silently rewrites links will eventually point something at the wrong page. Automate the finding and the reporting, and leave the decision with a person.
Picking a checker
Two open-source tools cover most needs:
- lychee is a fast link checker written in Rust. It checks links in files (Markdown, HTML) or on the URLs you pass it. It doesn't crawl recursively. Its own feature table marks website recursion as unsupported. That's fine if you give it the pages to check, and your XML sitemap already lists them.
- linkinator is a Node.js tool that can recurse through a live site with
--recurse, following same-domain links.
Desktop crawlers such as Screaming Frog also schedule crawls, but the free version is limited and scheduling is a paid feature. For a small business site, a free CI job is usually enough.
Option 1: lychee on a schedule, fed by your sitemap
Because lychee doesn't crawl, the trick is to pull page URLs from your sitemap and pass them in. Each URL is fetched and every link on it is checked:
# Extract page URLs from a simple (non-index) sitemap and check every link on them
URLS=$(curl -s https://www.example.com/sitemap.xml \
| grep -oE '<loc>[^<]+</loc>' | sed -E 's#</?loc>##g')
lychee --max-concurrency 8 \
--host-concurrency 2 \
--timeout 20 \
--max-retries 2 \
--cache --max-cache-age 1d \
--user-agent "ExampleLinkCheck/1.0 (+https://www.example.com/contact)" \
--format markdown --output lychee-report.md \
$URLS
If your sitemap is a sitemap index, fetch each child sitemap first. Run the XML sitemap checker if you're not sure which you have.
The GitHub Actions workflow
The official lychee-action is published as lycheeverse/lychee-action@v2. The workflow below runs every Monday at 06:00 UTC and opens an issue if anything fails:
name: Weekly link check
on:
workflow_dispatch:
schedule:
- cron: "0 6 * * 1"
jobs:
links:
runs-on: ubuntu-latest
permissions:
issues: write
steps:
- uses: actions/checkout@v5
- name: Collect URLs from sitemap
id: urls
run: |
LIST=$(curl -s https://www.example.com/sitemap.xml \
| grep -oE '<loc>[^<]+</loc>' | sed -E 's#</?loc>##g' | tr '\n' ' ')
echo "list=$LIST" >> "$GITHUB_OUTPUT"
- name: Restore lychee cache
uses: actions/cache@v4
with:
path: .lycheecache
key: lychee-${{ github.sha }}
restore-keys: lychee-
- name: Check links
id: lychee
uses: lycheeverse/lychee-action@v2
with:
args: >-
--cache --max-cache-age 1d
--max-concurrency 8 --host-concurrency 2
--max-retries 2
--no-progress
${{ steps.urls.outputs.list }}
fail: false
- name: Open an issue if links failed
if: steps.lychee.outputs.exit_code != 0
uses: peter-evans/create-issue-from-file@v5
with:
title: Broken link report
content-filepath: ./lychee/out.md
labels: report, automated issue
The page URLs are passed as inputs, so lychee fetches each page and checks the links on it. If you passed a text file of URLs instead, lychee would only check that those URLs respond, not the links on the pages. For security, the lychee-action README recommends pinning actions to a fixed version and letting Dependabot keep them updated.
Gotchas with GitHub's scheduler
From GitHub's documentation on scheduled events:
- Cron times are UTC. 06:00 UTC is 07:00 in London during British Summer Time and 02:00 in New York during daylight saving time.
- Scheduled runs can be delayed or dropped when GitHub is busy, especially on the hour. Pick an odd minute such as
17 6 * * 1. - Schedules only run from the default branch.
- In public repositories, scheduled workflows are disabled automatically after 60 days without repository activity. A link checker in a quiet repo will stop silently, so put a calendar reminder in to check it.
Option 2: linkinator for a full recursive crawl
If your sitemap is incomplete (which is itself worth fixing), crawl the site instead. On the command line:
npx linkinator https://www.example.com --recurse --concurrency 10 \
--skip "^https://www\.example\.com/wp-admin"
As a GitHub Action, linkinator-action is published as JustinBeckwith/linkinator-action@v2:
- uses: JustinBeckwith/linkinator-action@v2
with:
paths: "https://www.example.com"
recurse: true
concurrency: 10
skip: '^https://www\.example\.com/basket'
Exclude anything with side effects or infinite URL spaces: basket and checkout URLs, logout links, calendar pages that go on forever, and faceted search filters.
Checking external links politely
Your checker is a bot visiting someone else's server. A few rules keep you welcome and your results accurate:
- Keep concurrency low per host. lychee's
--host-concurrencyand--host-request-intervalexist for exactly this. - Cache results. With
--cache --max-cache-age 1d, a link checked yesterday isn't fetched again today. - Identify yourself. A user agent with a contact URL lets a site owner reach you instead of blocking you.
- Don't check externals on every deploy. Internal links on deploy, external links weekly or monthly.
- Treat some responses as "needs a human". Many sites answer bots with 403 Forbidden or 429 Too Many Requests while working fine in a browser. LinkedIn is known for returning a non-standard 999. Add these to an exclusion list (lychee reads a
.lycheeignorefile) or accept them, then spot-check them by hand now and then.
Only 404 and 410 on an external link reliably mean "gone". Timeouts and 5xx errors are often temporary, which is why retries are worth setting.
Fixing broken links at scale
A first automated run on an older site can return hundreds of failures. Work through them in this order.
1. Group by target, not by source
Fifty failures often come from one deleted page that's linked from your footer or a template. Sort the report by broken URL and fix the most-linked targets first. A template fix can clear dozens of rows at once.
2. Decide: redirect, update or remove
- Your own moved page: add a 301 redirect from the old URL to the closest equivalent. See 301 vs 302 redirects if you're unsure which code to use.
- Your own deleted page with no equivalent: let it return 404 or 410, and update internal links so they no longer point at it. Don't redirect everything to the homepage. Google tends to treat that as a soft 404.
- External page gone: link to its new location, an archived copy, or a better source, or remove the link.
3. Use bulk tools, carefully
On WordPress, WP-CLI can replace an old URL across the whole database:
# Always dry-run first and take a backup
wp search-replace 'https://www.example.com/old-services/' \
'https://www.example.com/services/' --dry-run
On Apache, maintain redirects in one place rather than scattering them. Our .htaccess guide shows the patterns. On Nginx, a map block keeps a long redirect list readable:
map $request_uri $redirect_target {
/old-services/ /services/;
/2019/price-list/ /pricing/;
}
server {
if ($redirect_target) { return 301 $redirect_target; }
}
4. Check the redirects you added
Every redirect you add is a chance to create a chain or a loop. Test a sample with the redirect checker to confirm each old URL lands on a 200 page in one hop. Then re-run your scheduled job by hand (workflow_dispatch) to confirm the report is clean.
Making it stick
The automation only helps if someone reads the output. Three habits help:
- One owner. Assign the issue to a named person, not "the team".
- A short report. Configure the checker to report failures only, so a clean week is an empty week.
- Spot checks in between. Before you publish a big page or migrate content, run the broken link checker on it by hand rather than waiting for Monday.
Broken links are one item in a wider set of checks that drift over time. Our website monitoring guide covers what else to watch. To schedule full audits next to link checks, see automating website audits.
FAQ
How often should I check my website for broken links?
Weekly suits most small business sites, with a manual check after big content changes or migrations. Sites that publish daily or have thousands of outbound links may want internal links checked on every deploy and external links weekly. Monthly is the minimum worth bothering with.
Can lychee crawl my whole website?
No. lychee checks links in the files or pages you give it, but it doesn't follow links recursively across a website. Feed it the URLs from your XML sitemap, or use a crawler such as linkinator with --recurse if your sitemap is incomplete.
Why does my link checker report links that work in my browser?
Many sites block or rate-limit automated requests and return 403, 429 or non-standard codes such as 999 to bots. The page is fine for humans. Add these domains to an exclusion list or accept those codes, and check them manually from time to time.
Is it rude to check external links automatically?
Not if you're polite about it. Use low per-host concurrency, cache results so you don't re-check the same link daily, identify your checker with a user agent that includes a contact URL, and check externals weekly or monthly rather than on every commit.
Do broken links hurt SEO?
A few broken outbound links won't sink a site, but broken internal links waste crawl effort, strand pages and frustrate visitors. Broken links to your own moved pages also throw away the value of the links pointing at them, which a 301 redirect would keep. Fixing them is low effort for a clear benefit.
Will my scheduled GitHub Action keep running forever?
Not necessarily. In public repositories GitHub disables scheduled workflows after 60 days without repository activity, and scheduled runs can be delayed during busy periods. Check now and then that the workflow is still running, or trigger it manually with workflow_dispatch.
Written by the MakeItFlow team
We build the website audit these guides are based on. Every fact is checked against the primary sources listed above and this guide was last reviewed on 25 October 2026 . How we research and check
Check your own site in 30 seconds
Run a free audit covering speed, SEO, DNS, email authentication and security headers — and get a prioritised fix list.
