The duplicate problem, and the one line that fixes it

Where the duplicates come from

Your automation calls something. Nothing comes back. It retries. The customer gets two invoices.

The uncomfortable part is that the automation did nothing wrong. When a call times out there are two possible worlds: the action never happened, or the action happened and the confirmation was lost on the way back. From where the caller is standing these look identical. There is no way to tell them apart by waiting.

So a retry is a coin toss between fixing a failure and causing a duplicate, unless you build the thing that makes the distinction unnecessary.

Idempotency, which is a plain idea with an ugly name

An action is idempotent if doing it twice has the same effect as doing it once.

Setting a customer’s status to “contacted” is idempotent. Set it twice, it is still contacted. Adding a row to a log is not. Sending an email is not. Charging a card is emphatically not.

For anything not naturally idempotent, you make it so by attaching a key that identifies the action, and checking that key before acting:

Before sending, ask: have I already sent message
  order-4471-ready-2026-09-14
If yes, stop. If no, send it and record the key.

The key is built from things that identify the specific action (the order number, the message type, the date) and never from a timestamp to the second or a random number, because those differ between the original and the retry, which is precisely the case you were trying to catch.

Most payment and messaging services accept an idempotency key directly in the request and will do this for you. Where yours does not, a two-column sheet of keys and timestamps is entirely sufficient for a small business.

Retries, done properly

Retry only errors that could plausibly be temporary: a timeout, a rate limit, a 500 from the far end. Never retry a rejection. If the supplier’s system said the product code is invalid, it will say so again, forever, faster than you can watch.

Wait longer between each attempt (one second, then four, then fifteen) and stop after three. A retry loop with no ceiling is the loop from the costs lesson, and it will do ten thousand attempts overnight without a word of complaint.

After the third failure, stop and tell a person. A queue of eleven things that failed is a normal Tuesday. Eleven things that silently did not happen is the problem this module exists to prevent.

The other duplicate: reprocessing

The second source is subtler. An automation that runs every hour and processes “new” items will reprocess everything if the definition of new breaks: a folder resynchronises, a timestamp resets, somebody restores a backup.

The defence is the same: record what you have already processed, by its own identifier, and skip it. Not “process everything since 10:00”, which depends on clocks, but “process everything I have not got a record of”, which depends on facts.

Test it on purpose

You will not discover a duplicate problem by accident at a convenient moment. Test it deliberately, once, before you trust the automation.

Run it twice on the same item. If two things happened, you have work to do. Then disconnect the network mid-run if you can, and see what the automation does when it comes back.

Ten minutes of this is worth more than any amount of careful reading, because the failure modes you are looking for are the ones that only appear when something is interrupted.

Why this belongs in a small-business course

Because the visible damage lands on customers. Two invoices for one job is not an engineering embarrassment; it is a phone call in which somebody decides whether your business is careless with money. Three identical booking reminders is not a glitch; it is the reason a client stops reading anything you send.

And because it is cheap to prevent. A key, a check, a list of what you already did. Ten lines in any automation tool, twenty minutes once, and the failure never happens again.

The short version

Before any automated action that a person will see or that touches money: what identifies this specific action, where is the record that it was done, and what does the automation do when it finds that record already there.

If you cannot answer all three, the automation is not ready to run unattended, however good its output looks.