{"course":{"slug":"ai-for-small-business","title":"AI for a Small Business","url":"https://addaly.com/learn/ai-for-small-business","level":"beginner","track":"work"},"lesson":{"slug":"the-duplicate-problem","title":"The duplicate problem, and the one line that fixes it","url":"https://addaly.com/learn/ai-for-small-business/the-duplicate-problem","position":63,"of":88,"minutes":8,"answer":"A retry cannot tell a failed action from a slow one, so every automated action needs a key that makes doing it twice the same as doing it once.","keyIdea":"A retry cannot tell a failed action from a slow one, so every automated action needs a key that makes doing it twice the same as doing it once."},"body":{"format":"markdown","text":"## Where the duplicates come from\n\nYour automation calls something. Nothing comes back. It retries. The customer gets two invoices.\n\nThe uncomfortable part is that the automation did nothing wrong. When a call times out there are two possible worlds: the action never happened, or the action happened and the confirmation was lost on the way back. From where the caller is standing these look identical. There is no way to tell them apart by waiting.\n\nSo a retry is a coin toss between fixing a failure and causing a duplicate, unless you build the thing that makes the distinction unnecessary.\n\n## Idempotency, which is a plain idea with an ugly name\n\nAn action is idempotent if doing it twice has the same effect as doing it once.\n\nSetting a customer's status to \"contacted\" is idempotent. Set it twice, it is still contacted. Adding a row to a log is not. Sending an email is not. Charging a card is emphatically not.\n\nFor anything not naturally idempotent, you make it so by attaching a key that identifies the action, and checking that key before acting:\n\n```text\nBefore sending, ask: have I already sent message\n  order-4471-ready-2026-09-14\nIf yes, stop. If no, send it and record the key.\n```\n\nThe key is built from things that identify the specific action (the order number, the message type, the date) and never from a timestamp to the second or a random number, because those differ between the original and the retry, which is precisely the case you were trying to catch.\n\nMost payment and messaging services accept an idempotency key directly in the request and will do this for you. Where yours does not, a two-column sheet of keys and timestamps is entirely sufficient for a small business.\n\n> Figure: The key that makes a retry safe. Steps: 1. Build the key (From what identifies this action: order 4471, message type ready, the date. Never a timestamp to the second or a random number, because those differ between the original and the retry.); 2. Check it first (Have I already done this one? A two-column sheet of keys and times is entirely sufficient for a small business.); 3. Act, then record (Do the thing and write the key down. Many payment and messaging services take the key in the request and do this for you.); 4. Retry only the temporary (A timeout, a rate limit, a 500 from the far end. Never a rejection: an invalid product code will be invalid again, forever, faster than you can watch.); 5. Stop after three, tell a person (One second, then four, then fifteen, then stop. Eleven things in a failed queue is a normal Tuesday; eleven things that silently did not happen is not.). When a call times out, the action either never happened or happened and lost its confirmation, and no amount of waiting tells you which. Two invoices for one job is a phone call about whether your business is careless with money.\n\n## Retries, done properly\n\nRetry only errors that could plausibly be temporary: a timeout, a rate limit, a 500 from the far end. Never retry a rejection. If the supplier's system said the product code is invalid, it will say so again, forever, faster than you can watch.\n\nWait longer between each attempt (one second, then four, then fifteen) and stop after three. A retry loop with no ceiling is the loop from the costs lesson, and it will do ten thousand attempts overnight without a word of complaint.\n\nAfter the third failure, stop and tell a person. A queue of eleven things that failed is a normal Tuesday. Eleven things that silently did not happen is the problem this module exists to prevent.\n\n## The other duplicate: reprocessing\n\nThe second source is subtler. An automation that runs every hour and processes \"new\" items will reprocess everything if the definition of new breaks: a folder resynchronises, a timestamp resets, somebody restores a backup.\n\nThe defence is the same: record what you have already processed, by its own identifier, and skip it. Not \"process everything since 10:00\", which depends on clocks, but \"process everything I have not got a record of\", which depends on facts.\n\n## Test it on purpose\n\nYou will not discover a duplicate problem by accident at a convenient moment. Test it deliberately, once, before you trust the automation.\n\nRun it twice on the same item. If two things happened, you have work to do. Then disconnect the network mid-run if you can, and see what the automation does when it comes back.\n\nTen minutes of this is worth more than any amount of careful reading, because the failure modes you are looking for are the ones that only appear when something is interrupted.\n\n## Why this belongs in a small-business course\n\nBecause the visible damage lands on customers. Two invoices for one job is not an engineering embarrassment; it is a phone call in which somebody decides whether your business is careless with money. Three identical booking reminders is not a glitch; it is the reason a client stops reading anything you send.\n\nAnd because it is cheap to prevent. A key, a check, a list of what you already did. Ten lines in any automation tool, twenty minutes once, and the failure never happens again.\n\n## The short version\n\nBefore any automated action that a person will see or that touches money: what identifies this specific action, where is the record that it was done, and what does the automation do when it finds that record already there.\n\nIf you cannot answer all three, the automation is not ready to run unattended, however good its output looks."},"check":{"question":"An automation sends order-ready texts and uses a key made of the order number plus the exact time of sending, checking that key before it sends. A timeout causes a retry four seconds later. What happens?","options":[{"text":"The retry is blocked, because the order number component of the key matches the record written by the first attempt.","correct":false,"tempting":"Assumes a partial match counts; the check compares whole keys, and the whole key differs."},{"text":"Nothing is sent at all, because the first attempt timed out before it could write its key, so the retry finds no record and correctly sends once.","correct":false,"tempting":"Describes one of the two possible worlds as if it were certain, which is exactly the ambiguity that makes timeouts hard."},{"text":"The customer may get two texts, because the time differs and so the two keys differ.","correct":true},{"text":"The retry succeeds and overwrites the earlier record, so the log shows a single send even though the customer received two messages and has no way to report the duplicate.","correct":false,"tempting":"Adds a plausible logging consequence on top of the real fault, but the record is not the thing that went wrong."}]},"prev":{"slug":"blast-radius","title":"Blast radius: what one confused run can reach","url":"https://addaly.com/learn/ai-for-small-business/blast-radius"},"next":{"slug":"a-log-you-can-read","title":"A log that answers \"why did it do that?\" three weeks later","url":"https://addaly.com/learn/ai-for-small-business/a-log-you-can-read"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The duplicate problem, and the one line that fixes it', in AI for a Small Business. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-for-small-business/the-duplicate-problem (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The duplicate problem, and the one line that fixes it","courseTitle":"AI for a Small Business","module":{"index":7,"slug":"letting-it-act","title":"Letting it act: automation you can trust with the keys"},"answer":null,"keyIdea":"A retry cannot tell a failed action from a slow one, so every automated action needs a key that makes doing it twice the same as doing it once.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"flow","steps":[{"label":"Build the key","detail":"From what identifies this action: order 4471, message type ready, the date. Never a timestamp to the second or a random number, because those differ between the original and the retry."},{"label":"Check it first","detail":"Have I already done this one? A two-column sheet of keys and times is entirely sufficient for a small business."},{"label":"Act, then record","detail":"Do the thing and write the key down. Many payment and messaging services take the key in the request and do this for you."},{"label":"Retry only the temporary","detail":"A timeout, a rate limit, a 500 from the far end. Never a rejection: an invalid product code will be invalid again, forever, faster than you can watch."},{"label":"Stop after three, tell a person","detail":"One second, then four, then fifteen, then stop. Eleven things in a failed queue is a normal Tuesday; eleven things that silently did not happen is not."}],"title":"The key that makes a retry safe","caption":"When a call times out, the action either never happened or happened and lost its confirmation, and no amount of waiting tells you which. Two invoices for one job is a phone call about whether your business is careless with money."},"alt":"Figure: The key that makes a retry safe. Steps: 1. Build the key (From what identifies this action: order 4471, message type ready, the date. Never a timestamp to the second or a random number, because those differ between the original and the retry.); 2. Check it first (Have I already done this one? A two-column sheet of keys and times is entirely sufficient for a small business.); 3. Act, then record (Do the thing and write the key down. Many payment and messaging services take the key in the request and do this for you.); 4. Retry only the temporary (A timeout, a rate limit, a 500 from the far end. Never a rejection: an invalid product code will be invalid again, forever, faster than you can watch.); 5. Stop after three, tell a person (One second, then four, then fifteen, then stop. Eleven things in a failed queue is a normal Tuesday; eleven things that silently did not happen is not.). When a call times out, the action either never happened or happened and lost its confirmation, and no amount of waiting tells you which. Two invoices for one job is a phone call about whether your business is careless with money."}},"blocks":[{"t":"h2","id":"where-the-duplicates-come-from","runs":[{"text":"Where the duplicates come from"}]},{"t":"p","runs":[{"text":"Your automation calls something. Nothing comes back. It retries. The customer gets two invoices."}]},{"t":"p","runs":[{"text":"The uncomfortable part is that the automation did nothing wrong. When a call times out there are two possible worlds: the action never happened, or the action happened and the confirmation was lost on the way back. From where the caller is standing these look identical. There is no way to tell them apart by waiting."}]},{"t":"p","runs":[{"text":"So a retry is a coin toss between fixing a failure and causing a duplicate, unless you build the thing that makes the distinction unnecessary."}]},{"t":"h2","id":"idempotency-which-is-a-plain-idea-with-an-ugly-name","runs":[{"text":"Idempotency, which is a plain idea with an ugly name"}]},{"t":"p","runs":[{"text":"An action is idempotent if doing it twice has the same effect as doing it once."}]},{"t":"p","runs":[{"text":"Setting a customer’s status to “contacted” is idempotent. Set it twice, it is still contacted. Adding a row to a log is not. Sending an email is not. Charging a card is emphatically not."}]},{"t":"p","runs":[{"text":"For anything not naturally idempotent, you make it so by attaching a key that identifies the action, and checking that key before acting:"}]},{"t":"code","lang":"text","label":"Text","text":"Before sending, ask: have I already sent message\n  order-4471-ready-2026-09-14\nIf yes, stop. If no, send it and record the key."},{"t":"p","runs":[{"text":"The key is built from things that identify the specific action (the order number, the message type, the date) and never from a timestamp to the second or a random number, because those differ between the original and the retry, which is precisely the case you were trying to catch."}]},{"t":"p","runs":[{"text":"Most payment and messaging services accept an idempotency key directly in the request and will do this for you. Where yours does not, a two-column sheet of keys and timestamps is entirely sufficient for a small business."}]},{"t":"h2","id":"retries-done-properly","runs":[{"text":"Retries, done properly"}]},{"t":"p","runs":[{"text":"Retry only errors that could plausibly be temporary: a timeout, a rate limit, a 500 from the far end. Never retry a rejection. If the supplier’s system said the product code is invalid, it will say so again, forever, faster than you can watch."}]},{"t":"p","runs":[{"text":"Wait longer between each attempt (one second, then four, then fifteen) and stop after three. A retry loop with no ceiling is the loop from the costs lesson, and it will do ten thousand attempts overnight without a word of complaint."}]},{"t":"p","runs":[{"text":"After the third failure, stop and tell a person. A queue of eleven things that failed is a normal Tuesday. Eleven things that silently did not happen is the problem this module exists to prevent."}]},{"t":"h2","id":"the-other-duplicate-reprocessing","runs":[{"text":"The other duplicate: reprocessing"}]},{"t":"p","runs":[{"text":"The second source is subtler. An automation that runs every hour and processes “new” items will reprocess everything if the definition of new breaks: a folder resynchronises, a timestamp resets, somebody restores a backup."}]},{"t":"p","runs":[{"text":"The defence is the same: record what you have already processed, by its own identifier, and skip it. Not “process everything since 10:00”, which depends on clocks, but “process everything I have not got a record of”, which depends on facts."}]},{"t":"h2","id":"test-it-on-purpose","runs":[{"text":"Test it on purpose"}]},{"t":"p","runs":[{"text":"You will not discover a duplicate problem by accident at a convenient moment. Test it deliberately, once, before you trust the automation."}]},{"t":"p","runs":[{"text":"Run it twice on the same item. If two things happened, you have work to do. Then disconnect the network mid-run if you can, and see what the automation does when it comes back."}]},{"t":"p","runs":[{"text":"Ten minutes of this is worth more than any amount of careful reading, because the failure modes you are looking for are the ones that only appear when something is interrupted."}]},{"t":"h2","id":"why-this-belongs-in-a-small-business-course","runs":[{"text":"Why this belongs in a small-business course"}]},{"t":"p","runs":[{"text":"Because the visible damage lands on customers. Two invoices for one job is not an engineering embarrassment; it is a phone call in which somebody decides whether your business is careless with money. Three identical booking reminders is not a glitch; it is the reason a client stops reading anything you send."}]},{"t":"p","runs":[{"text":"And because it is cheap to prevent. A key, a check, a list of what you already did. Ten lines in any automation tool, twenty minutes once, and the failure never happens again."}]},{"t":"h2","id":"the-short-version","runs":[{"text":"The short version"}]},{"t":"p","runs":[{"text":"Before any automated action that a person will see or that touches money: what identifies this specific action, where is the record that it was done, and what does the automation do when it finds that record already there."}]},{"t":"p","runs":[{"text":"If you cannot answer all three, the automation is not ready to run unattended, however good its output looks."}]}],"check":{"question":[{"text":"An automation sends order-ready texts and uses a key made of the order number plus the exact time of sending, checking that key before it sends. A timeout causes a retry four seconds later. What happens?"}],"options":[{"text":[{"text":"The retry is blocked, because the order number component of the key matches the record written by the first attempt."}],"correct":false,"tempting":[{"text":"Assumes a partial match counts; the check compares whole keys, and the whole key differs."}]},{"text":[{"text":"Nothing is sent at all, because the first attempt timed out before it could write its key, so the retry finds no record and correctly sends once."}],"correct":false,"tempting":[{"text":"Describes one of the two possible worlds as if it were certain, which is exactly the ambiguity that makes timeouts hard."}]},{"text":[{"text":"The customer may get two texts, because the time differs and so the two keys differ."}],"correct":true,"tempting":null},{"text":[{"text":"The retry succeeds and overwrites the earlier record, so the log shows a single send even though the customer received two messages and has no way to report the duplicate."}],"correct":false,"tempting":[{"text":"Adds a plausible logging consequence on top of the real fault, but the record is not the thing that went wrong."}]}],"order":[0,2,1,3],"why":null},"prev":{"slug":"blast-radius","title":"Blast radius: what one confused run can reach","minutes":9},"next":{"slug":"a-log-you-can-read","title":"A log that answers “why did it do that?” three weeks later","minutes":8}}}