How do you set up a broken link check that runs itself?

How do I set up a broken link check that runs itself?

Take your sitemap as the source of truth, request each URL on a schedule and after every publish, record the status codes, and alert only when something that used to work stops working. The whole thing is a few moving parts and it replaces a job nobody remembers to do.

Broken links are the purest example of silent failure on a website. Nothing warns you. The page just stops resolving, and the first person to find out is a reader who does not tell you, or a crawler that quietly stops trusting your site.

This is the setup I would actually build, in order, for someone who owns a CMS site and does not want a monitoring subscription.

What counts as a broken link worth alerting on?

Something that used to return a page and now does not. A URL returning not found after previously working is an alert. A URL that has never existed is a typo somewhere, which is different and usually less urgent.

That distinction is what separates a useful monitor from a noisy one. If you alert on every non success status you will get a stream of noise from links in old comments, mistyped external references and pages you deliberately removed. If you alert on transitions, you get a short list of real regressions.

Redirects need their own treatment. A permanent redirect is not broken, it is working as designed, but a chain of several redirects is a slow problem worth knowing about. Server errors are the most urgent category of all, because they usually mean something broader is wrong than one link.

Where does the list of URLs come from?

Two lists, and you need both. Your sitemap tells you what should exist. A crawl tells you what is actually linked. The union is what you check, and the difference between them is itself informative.

Start with the sitemap because it is machine readable and free. Pull it, extract the location values, and store them. That is your baseline set and it updates itself whenever you publish, which is exactly the property you want in an automated check.

For the links inside your pages, a periodic crawl is the practical answer. Screaming Frog's own site states that its free version crawls up to 500 URLs in a single crawl, which covers a lot of content sites and costs nothing. That gives you the outbound and internal link inventory that a sitemap cannot. I have set out the manual version of this in how to audit internal links on a 500 page CMS site.

How do you check them without hammering your own site?

Request one URL at a time with a small pause between them, and ask for headers rather than whole pages where you can. A checker that fires hundreds of parallel requests at your own hosting is indistinguishable from a small attack.

Pacing also makes the results more trustworthy. A burst of requests can produce timeouts that look like failures but are really rate limiting, and then you spend an afternoon investigating a problem you created. Slow and boring gives you a clean signal.

Build a dry run mode before you build the alerting. Run the whole check, write the results somewhere, and send nothing for the first week. That is the only way to discover what your baseline noise looks like, and it is a habit I apply to everything I build, for reasons I have set out in why every automation I build has a dry run mode.

What should trigger the check?

Two triggers: a schedule, and a publish. The schedule catches decay from outside your site. The publish catches the damage you just did yourself, which is the more common cause.

On a Webflow site the publish trigger is available directly. Webflow's webhook interface supports a site publish trigger, alongside triggers for collection items being created, changed, deleted, published and unpublished. Subscribing to site publish and starting a check a few minutes later catches the classic mistake of renaming a slug and forgetting what linked to it.

There is room for plenty of these. Webflow's interface documents a limit of seventy five registrations per trigger type per site, which is far more than anyone needs for this purpose, so you do not have to be precious about adding a dedicated webhook for link checking.

Where should the results go?

Into a store that keeps history, and into a message only when the state changes. History is what lets you distinguish a genuine regression from a URL that has been failing for months.

One row per URL per run in a sheet or a small table is enough. You are storing the URL, the status, and the timestamp. That is it. Once you have two runs you can diff them, and the diff is the alert.

The message itself should contain the actual URLs, not a link to a dashboard. Somebody reading it on a phone should be able to see which page broke without opening anything. Anything requiring a click will be looked at tomorrow, which defeats the purpose of monitoring.

How do you avoid alert fatigue?

Alert on change, suppress repeats, and give yourself a way to acknowledge something. Those three rules do almost all the work.

Alerting on change means a URL that broke yesterday and is still broken today generates nothing new. Suppressing repeats means a flaky external link does not wake you up twice a week. Acknowledgement means you can mark a known issue as known, so it stops competing for attention with new problems.

The one thing I would never do is send a daily summary that says everything is fine. Those get filtered within a fortnight, and then the day it says something is wrong, nobody reads it. Silence when healthy is a feature.

What does it cost to run?

It depends entirely on how you count requests. If you build it on a platform that bills per action, the arithmetic matters: Zapier's own documentation defines a task as any successful action that runs in Zapier, and a check that tests several hundred URLs one at a time would be several hundred actions per run.

That is the reason most people build this part as a small script rather than as a visual workflow. Checking five hundred URLs in a loop inside one step is cheap. Checking five hundred URLs as five hundred steps is not. Use the automation platform for the trigger and the alerting, and do the loop somewhere that does not charge per iteration.

Check current rates on the vendor's own documentation before you design around any of this, because billing details change and I am describing what the page says today.

What does this not catch?

Quite a lot, and it is worth being honest about it. A URL that returns a successful status but shows an error message inside the page will pass. So will a page that loads but has lost its content. Status codes are a shallow check.

It also will not catch links that are wrong rather than broken. A link that resolves successfully but points at the wrong article is invisible to any status based monitor, and that class of error is more common than people think.

And it does nothing about what happens to the visitor who hits a genuine dead end. Monitoring tells you about the problem; the 404 page is what protects the reader while you fix it, which is a separate design job I have written about in what a good 404 page does on a content heavy site.

What should you do next?

Pull your sitemap into a sheet today and check the status of every URL once, by hand or with a free crawl. That single pass will usually find something, and it tells you whether you have a monitoring problem or an existing mess to clean up first.

Then build the smallest automated version: one scheduled run, results appended to the same sheet, and a message only when a previously working URL fails. Add the publish trigger once the scheduled version has been quiet for a week.

If you want a second opinion on where the fragile links in your site actually are, reach out. Across more than 70 projects for more than 25 clients over more than six years, the answer has almost always been slug changes rather than anything external.