When Should You Turn an Automation Off?

When should you turn an automation off?

When verifying its output costs more than doing the work would have. That is the whole test, and it catches more automations than any other question I know. If you find yourself checking every result before you trust it, the automation has not removed the work, it has moved it and added a step.

I have turned off things I built, which is an uncomfortable thing to write when automation is part of what I sell. But I would rather be honest about it, because the alternative is the version of this advice where everything is worth automating and nothing ever gets retired.

I still run plenty. The Ajust pipeline on Airtable and WhaleSync has delivered more than 25,000 cases, helped over 400,000 people, and saved upward of 50,000 hours. Kismet Health runs on HubSpot through Zapier. Those earn their keep. The point of this article is that some things do not, and the failure is usually visible in advance.

What does an automation actually cost?

Four things, and most people count one. There is the build, which is the only cost anyone estimates. There is the run cost, in subscriptions and tasks and tokens. There is the verify cost, which is the time somebody spends confirming the output is right. And there is the repair cost, which arrives on a schedule nobody chooses.

Verify is the one that quietly dominates, and it is the one nobody puts in the business case. An automation that produces something a human has to read carefully before sending has not saved the reading. It has saved the typing, which is usually the cheap part.

So the honest arithmetic is not whether the automation is faster than doing it by hand. It is whether the build, plus the run, plus the verify, plus the repair, is less than doing it by hand for as long as you intend to keep it. Stated that way, a surprising number of candidates fail.

What kind of automation fails this test most often?

Anything that produces output for a human audience where the stakes of being wrong are high and the variation between cases is real. Outbound messages. Client updates. Anything that gets a person's name in it.

The pattern is consistent. The automation handles the ninety percent case perfectly, and the ten percent where context matters is exactly the ten percent where sending something slightly wrong is expensive. So you check all of it to catch the tenth, and now you are reading every output to catch the small number that need judgement.

What makes this hard to see in advance is that the demo always works. You test it on three cases, all of which are the ninety percent, and conclude it works. The failure mode only appears at volume, which is after you have committed.

What is the difference between repetitive and merely frequent?

Repetitive means the same inputs produce the same correct output every time. Frequent means it happens a lot. They feel identical from inside a busy week and they are completely different automation candidates.

A frequent task that requires a fresh judgement each time is not repetitive, it is just common. Automating it does not remove the judgement, it hides it, and hidden judgement is worse than visible judgement because nobody reviews it.

The question I ask now is whether I could write down the rule completely, with no exceptions clause ending in it depends. If the rule needs a human to arbitrate the edges, automate the mechanical part and leave the arbitration visible. Half an automation with an honest handoff beats a whole one that quietly guesses.

How do you tell whether it is the tool or the design?

Ask whether a different tool would have produced a different outcome. In almost every case I have looked at, the answer is no, and that is the point. The automations I have retired were not defeated by Zapier or Make or n8n or anything I built with Claude Code. They were defeated by the task being a worse fit for automation than it looked.

That distinction matters because the instinct when something is not working is to rebuild it somewhere else. Moving a badly specified workflow from one platform to another gives you a second build cost and the same problem, dressed in a new interface.

So before rebuilding, write down what the automation is supposed to guarantee. If you cannot state the guarantee in one sentence, the problem is upstream of the tool, and no migration fixes it.

What is the reversal test?

Imagine the automation is off tomorrow morning. Ask what breaks, who notices, and how long it takes to notice. Then ask what you would actually do instead.

If the answer is that somebody does the task manually in fifteen minutes and nothing else changes, you have learned what the automation is worth. If the answer is that a process quietly stops and nobody finds out for a week, you have learned something more important, which is that you have an automation with no monitoring attached to a task that matters.

Run this on your automations occasionally, not because you plan to turn anything off, but because the answers tell you where your monitoring should be. It is the same instinct behind watching for silent automation failures, approached from the other direction.

What do you do with the half that was worth keeping?

Keep it, and be explicit about where it stops. Most automations I have retired were not entirely wrong. The data gathering was fine and the composing was the problem, or the routing was fine and the decision about who to route to was the problem.

So the useful move is rarely delete. It is trim. Let the automation do the part that has a complete rule, have it stop and hand over cleanly at the point where judgement starts, and make that handoff a real artefact rather than an implied one. A draft in a folder with the context attached is enormously useful. The same draft sent automatically is a liability.

That reframing has changed how I build. I now design for the handoff first and treat full automation as the thing I earn later, once the rule has proven complete over enough real cases to trust it.

How do you turn something off without losing the work?

Disable rather than delete, and write down why. A disabled workflow costs nothing and keeps the logic, the field mappings, and the small decisions you will not remember in six months. A deleted one costs you all of that, and the idea will come back.

The note matters more than the workflow. Say what it did, why you turned it off, and what would have to change for it to be worth turning back on. That last part is the valuable sentence, because it converts a failure into a condition you can check later.

Also tell whoever depended on it, before they discover it. An automation quietly stopping is indistinguishable from an automation quietly breaking, and the second one destroys trust in everything else you have built. This is the same principle as the decay problem I described in what breaks when an automation runs unattended, where silence is always read as working.

Does this mean you should automate less?

No. It means you should automate the parts with complete rules and stop pretending the rest is a tooling problem. Across six years and more than 70 projects the automations that survived are the boring ones: moving records, syncing fields, routing by an unambiguous condition, generating a draft somebody reviews.

The ones that did not survive were the ambitious ones, and they were ambitious in a specific way. They tried to automate the decision rather than the work around the decision, which is where the judgement is and where a person should stay.

I would rather run five automations I never think about than fifteen I check. The first set compounds. The second set is a second job, and I already have one. The same discipline applies when choosing whether to move something out of a visual builder and into code, which is a question about the rule, not the runtime.

What should you do next?

List everything you have running and put one word next to each: trusted, or checked. Anything in the checked column is a candidate, and the question for each is whether the checking is temporary while you build confidence, or permanent because the task needs judgement.

For the permanent ones, trim rather than delete. Find the point where the complete rule ends, stop there, and make the handoff explicit. You will usually keep most of the value and lose all of the anxiety.

And if you have something running that you no longer trust and cannot decide whether to fix, trim, or retire, reach out. Deciding that from outside is much easier than deciding it about something you built.