What Should an Automation Do When It Is Not Sure?

What should an automation do when it is not sure?

Stop and hold the item, rather than guess and continue. Most workflows are built with only two outcomes, success and failure, and almost every expensive automation problem I have seen lives in the third state nobody designed: the run that completed while being wrong.

A failure is loud and gets fixed. A confident mistake gets written to your CRM, syncs to three other systems, and is discovered by a customer six weeks later.

So the design question is not how to make an automation more accurate. It is what it does on the occasions when accuracy is not available.

Why is guess and continue the default?

Because it is what every tool does unless you intervene. A mapping step receives a value it does not recognise and passes something through. A model asked to classify an ambiguous record picks the nearest option. Nothing in the design says that uncertainty is a state worth representing.

It is also the default because holding items feels like a failure of the automation. Somebody built this to remove manual work, and a queue of held records looks like the manual work coming back.

That framing is backwards. The held items are the ones that always needed a person. The automation has just told you which ones they are, which is a considerable service.

What does the vendor guidance actually say?

For the AI parts of a workflow, it is remarkably direct about not bluffing. Anthropic's prompting guidance includes a sample instruction telling the model never to speculate about code it has not opened, and never to make claims before investigating unless it is certain of the correct answer, in order to give grounded answers.

Generalise that past code and you have your rule. An automation should not assert something it has not checked, and where the check is not possible, the honest output is a hold rather than a best guess wearing a confident tone.

The same guidance pairs it with an action rule: consider reversibility and impact, take local reversible actions freely, and ask before anything hard to reverse, shared, or destructive. Uncertainty plus irreversibility is the combination that should always stop a run.

How do you define unsure when there is no AI in the step?

By listing the conditions where your logic is guessing. You do not need a confidence score to know that a record with a missing email, an unrecognised company name, or a date that parsed into next century is not something you should act on.

In practice I write these as explicit hold conditions alongside the error conditions. Field empty where it has never been empty before. Value not in the known set. Count wildly outside the normal range for this time of day. Any one of them sends the item to the hold queue instead of the next step.

The unrecognised value case is the one that catches the most real problems. A new option added in an upstream tool that your mapping does not know about will otherwise be silently coerced into whatever your fallback is, forever.

Where should an uncertain item go?

Into a visible queue with the full context attached, not into a log file. The point of holding is that a person will decide, and a person cannot decide from a row that says validation failed. Give them the whole record and the reason it stopped.

What I attach to a held item is the incoming record, the step that stopped, the specific condition that tripped, and what the automation would have done if it had continued. That last field is the one that makes the review fast, because the reviewer is approving or rejecting a proposed action rather than starting from nothing.

For content work the queue and the review step can be the same thing, which is a neat simplification. I set out that version in setting up a review step before CMS items go live.

Who gets told, and how?

One named person, through a channel they actually watch, and only when the hold needs a decision today. Google's Site Reliability Engineering guidance applies here even though it is written about pagers: every page should be actionable, and every page response should require intelligence, because a merely robotic response should not have been a page.

Run that test on your hold notifications. If the person's response to a held item is always the same click, that is not a decision, and you should encode the rule instead of asking a human to perform it.

Do not route them to email. The same SRE guidance says email alerts are of very limited value and tend to easily become overrun with noise, which matches exactly what happens to a held items inbox after a fortnight.

How do you stop the queue becoming a graveyard?

Give every held item an owner and an expiry. An item nobody has looked at in a week is not waiting for review, it is lost, and pretending otherwise is how a safety mechanism quietly becomes a liability of its own.

I set an explicit rule for what happens at expiry, and it differs by workflow. Some items should be discarded with a note. Some should escalate. A few genuinely should proceed with the fallback, and that is a legitimate decision as long as somebody made it in advance rather than by neglect.

Watch the queue size as a metric in its own right. A queue that grows steadily means your hold conditions are too broad or your upstream data has changed shape, and both of those are worth knowing before the queue becomes unusable.

What does holding actually cost you?

Latency, and that is usually the right trade. An item that waits four hours for a person is slower than an item processed instantly and wrongly, and only one of those two ends in a cleanup project you have to explain to someone.

The cost is real in specific cases though, and worth naming. If your automation replies to inbound enquiries, a hold means a slower reply, and slower replies cost deals. In that situation the answer is not to remove the hold but to narrow it, so that only genuinely ambiguous cases wait.

The automations I run for Ajust move case data between Airtable and Webflow through WhaleSync and have delivered more than 25,000 cases. At that volume, a hold rate of even a few percent is a meaningful queue, which is exactly why the conditions have to be specific rather than cautious.

How does this interact with retries?

They solve different problems and should not be confused. A retry is for a step that failed for a reason that might go away, such as a timeout or a rate limit. A hold is for a step that succeeded technically while producing something you do not trust.

Retrying an uncertain result just produces the same uncertain result more times, and on a metered API it does so at cost. I went through the mechanics of the retry side in handling rate limits and retries in a marketing automation.

Keep the two paths separate in the code as well as in your head. When something goes wrong at three in the morning, you want to be able to tell instantly whether the system could not act or chose not to.

What should you do next?

Take your most important automation and write down the three situations where it is guessing. You will know them already, because they are the cases you check manually every so often without quite admitting that is what you are doing.

Then turn one of those into a hold with a visible queue and a named owner. One condition, one queue, one person. That is an afternoon, and it converts a silent risk into a short list somebody reads.

After six years of running these systems, the automations I trust are not the ones that never stop. They are the ones that stop for the right reasons and tell me why. If you have a workflow that has never once refused to act, reach out and let's chat about where it is guessing. The harder question of what it should never be allowed to do at all is one I covered in what an automation should never be allowed to do.