How Do You Retry a Failed Webhook Without Creating Duplicates?

Why does retrying a failed webhook create duplicates?

Because the sender often cannot tell whether your system already did the work. If your endpoint processed an event but responded too slowly or returned an error, the sender sends the same event again. Without a check, your automation runs twice and you get two contacts, two deals, or two welcome emails from one real event.

This is one of the most common silent failures in marketing and sales automation. Nothing looks broken. The workflow history shows green runs. Then a sales rep notices two copies of the same lead in HubSpot, or a customer asks why they got the same onboarding email twice in one minute.

I run automations in production for clients, including the Airtable and WhaleSync system behind Ajust and the HubSpot and Zapier setup for Kismet Health. The fix for duplicates is the same in every stack I work with. You design the receiving step so that running it twice gives the same result as running it once.

What does idempotent mean for an automation?

An idempotent step produces the same end state no matter how many times it runs with the same input. Creating a contact is not idempotent by default, because each run makes a new one. Creating a contact only if no contact with that email exists is idempotent, because the second run finds the first and stops.

The word comes from engineering, but the idea is simple for anyone who builds in Zapier, Make, or n8n. Ask one question about every step that writes data. If this step ran twice with the same event, would anything bad happen? If the answer is yes, that step needs a guard.

Stripe's API documentation gives a clean example of the concept. It describes idempotency as a way to safely retry requests "without accidentally performing the same operation twice", using a unique key that lets the server recognize a repeat of the same request. You can apply that same pattern inside your own automations.

How do senders like HubSpot retry webhooks?

Each platform has its own retry rules, so read the sender's documentation. HubSpot's developer docs say webhook notifications are retried up to 10 times, spread over the next 24 hours. Retries happen when HubSpot cannot connect, when your service takes longer than 5 seconds to respond, or when it returns a 4xx or 5xx status code.

That 5-second window is the detail that catches people. Suppose your endpoint receives a HubSpot event, kicks off a slow enrichment call, and only then responds. If that takes longer than 5 seconds, HubSpot treats it as a failure and sends the event again, even though your workflow is still running the first copy.

The lesson applies far beyond HubSpot. Any sender with retries assumes your endpoint might fail, so it errs on the side of sending again. Your job is to make "again" harmless. Read the retry rules for every tool that sends you events, and write them into your automation documentation.

What is the simplest way to stop duplicate records?

Search before you create. Before any step that creates a record, look up whether a matching record already exists using a stable identifier such as email, domain, or an external ID. If it exists, update it. If it does not, create it. This search-then-write pattern prevents most duplicate contacts and companies on its own.

Most automation tools give you the pieces for this. In Zapier, a search step placed before a create step does the job. In Make and n8n, a search module followed by a router or an IF node does the same. Check your CRM documentation for how it matches records natively. The building blocks exist. They just need to be used every time.

The weak spot is choosing the identifier. Email works for contacts. Domain works for companies, with care for subsidiaries and personal email domains. For deals, there is rarely a natural key, which is why deals duplicate more often than contacts. That is where the next technique comes in.

How do you use an event ID as an idempotency key?

Store the unique ID of every event you process, and check it before doing any work. When an event arrives, look up its ID in a log table. If the ID is already there, stop and return success. If not, record the ID and continue. Retries of the same event then become harmless no-ops.

Many senders include a unique identifier for each event or delivery. When they do not, build your own key from fields that together identify the event, such as the object ID, the event type, and the timestamp of the change. The key needs to be the same for a retry and different for a genuinely new event.

I use an Airtable table or a small database table for the log, depending on the volume. The table has the key, the time it was processed, and the outcome. That same table doubles as a run history, which I find invaluable later. I covered that idea in more depth in how to version an automation so you can audit runs.

Should your endpoint respond before or after it does the work?

Respond fast, then work. Acknowledge the webhook as soon as you have safely stored the event, and do the slow processing afterward. This keeps you inside the sender's timeout, which cuts unnecessary retries. Combined with an event ID check, it means a retry only happens when something truly failed, and even then it is safe.

In practice, that means splitting one automation into two parts. The first part receives the event, writes it to a queue or log table, and responds. The second part picks up stored events and runs the enrichment, the CRM writes, and the notifications. If the second part fails, you can rerun it without asking the sender for anything.

No-code tools vary in how much control they give you over the response. Some send the acknowledgement automatically when the trigger fires. Others let you return a response at a chosen step. Check how your tool behaves before you rely on this pattern, and test it with a deliberately slow step.

What should happen when a retry finds partial work?

Design every step to check its own result, not just the first step. If a run created the contact but failed before creating the deal, a retry should skip the contact, find it, and continue to the deal. Each write step searches before it creates, so a retry picks up where the failed run stopped.

This is the difference between a guard at the door and a guard at every room. The event ID check stops full duplicates. Per-step checks handle the messier case where a run half-finished. You need both, because real failures rarely happen at a tidy point in the workflow.

The same thinking matters when two tools write to the same system. If a form tool and an enrichment tool both create contacts, deduplication has to account for both. I wrote about the ownership side of this in which tool should own a field in a two-way sync.

When should a retry stop and alert a human instead?

Stop retrying when the error will not fix itself. A timeout or a rate limit is worth retrying. A missing required field, a deleted record, or a permission error is not. Retrying those just repeats the failure. Route them to a human with the event details so someone can decide what to do.

HubSpot's own docs reflect this split for workflows, noting that workflows will not retry after 4xx responses, with an exception for 429 rate limit errors. That is a sensible default for your own automations too. Retry what is temporary. Escalate what is structural.

I keep the escalation simple: a Slack message or an email to the owner, with the event key, the step that failed, and the error text. My longer take on this trade-off is in when an automation should fail loudly versus retry.

How do you test that your retries are safe?

Send the same event twice on purpose. Trigger a test event, let the automation finish, then replay the exact same payload. Check that nothing new was created and no message went out twice. Then simulate a failure midway, rerun it, and confirm the workflow finishes cleanly from where it stopped.

Most teams test the happy path once and ship. Retry testing takes an extra half hour and catches the bugs that cost the most trust, such as double emails to customers or inflated pipeline numbers. I treat a passing replay test as a requirement before any webhook automation goes live.

Keep the test payloads. When you change the automation later, rerun the same replay tests. A change that quietly breaks idempotency is easy to make and hard to spot without them.

What should you do next?

Pick your most important webhook automation and audit it. List every step that writes data, add a search-before-create guard to each, and add an event ID log at the start. Move slow work after the response. Then run a replay test. One hardened automation teaches you the pattern for the rest.

Duplicates are not a cosmetic problem. They break lead routing, inflate reports, and annoy customers. Fixing them at the design level is far cheaper than cleaning up a CRM every quarter.

If you want a second pair of eyes on a workflow that keeps creating duplicates, reach out. Making automations safe to retry is a big part of the work I do.