{"course":{"slug":"ai-for-small-business","title":"AI for a Small Business","url":"https://addaly.com/learn/ai-for-small-business","level":"beginner","track":"work"},"lesson":{"slug":"a-log-you-can-read","title":"A log that answers \"why did it do that?\" three weeks later","url":"https://addaly.com/learn/ai-for-small-business/a-log-you-can-read","position":64,"of":88,"minutes":8,"answer":"A run is only defensible if you kept the input, the output, the decision and the version of the prompt and model that produced it.","keyIdea":"A run is only defensible if you kept the input, the output, the decision and the version of the prompt and model that produced it, because a model's output is not reproducible from the prompt alone."},"body":{"format":"markdown","text":"## The question you will eventually be asked\n\nA customer forwards you a message your automation sent five weeks ago and asks why they were told that.\n\nYou open the automation. The prompt has been edited twice since. The provider has upgraded the model underneath you. You re-run it with the same input and get something different and perfectly reasonable.\n\nYou now cannot answer the question. Not because you were careless with the automation, but because you did not keep a record, and a model's output is not recoverable from the prompt: same input, same instructions, different day, different words.\n\n## What to record, per run\n\nSix things, and none of them is expensive:\n\n- **When**, with a time zone.\n- **What went in**: the actual customer message or document, not a summary of it.\n- **What came out**: the full text or the full set of fields.\n- **What was decided**: which branch it took, what it sent, to whom, and whether a person approved.\n- **Which prompt version**: a version number you increment whenever you edit it, and the prompt text stored against that number.\n- **Which model**, by its exact name including the version suffix.\n\nAdd cost and duration if it is free to do so; both are useful later for spotting a change nobody announced.\n\n## Where it can live\n\nA spreadsheet. Genuinely. A row per run, one tab per automation, on a business that does a few hundred runs a month. Every automation platform can append a row to a sheet, and a sheet is searchable, sortable and readable by everybody in the business without a login to anything technical.\n\nBeyond a few thousand rows a month, a proper database is easier, but almost no small business reaches that with AI automations, and the sheet you will actually keep beats the system you would have to maintain.\n\nFree path throughout: Google Sheets with Apps Script, LibreOffice with a CSV appended by a script, or a plain text file with one JSON object per line. The last is what most logging systems are underneath.\n\n## Why the prompt version is the field people skip\n\nBecause it feels redundant. The prompt is right there in the tool.\n\nIt is right there in its *current* state. Small-business prompts get edited constantly: a customer complains about a phrase, somebody adds a rule, the price changes. Without a version stamp, every run before the last edit is unexplainable, and the one thing you most want to know when something goes wrong is whether it was caused by the edit you made on Tuesday.\n\nIncrement a number in the prompt itself. `v7`. Log the number. Keep the old text in the same file. This is the cheapest useful discipline in the whole module.\n\n## Redaction, briefly\n\nYour log now contains customer messages, which means it is customer data, which means module nine's obligations apply to it as much as to your CRM. Two practical consequences: do not keep it forever (ninety days covers almost every question you will actually be asked) and do not put it somewhere with looser access than the system the data came from.\n\nIf the inputs contain anything sensitive, log a reference to the record rather than the content itself, and accept that you have traded some diagnostic power for a smaller problem.\n\n## What you will use it for, apart from complaints\n\n**Spotting a silent change.** Average output length drops by a third one Wednesday. Nobody told you the model was updated; the log did.\n\n**Finding the real error rate.** Sample twenty logged runs a month and read them properly. This is how you discover the automation has been quietly mishandling one category since May.\n\n**Deciding whether to climb the ladder.** The evidence from the first lesson in this module (fifty consecutive clean items) is a claim about your log. Without one, it is a feeling.\n\n## The test\n\nPick a run from a month ago at random. Can you say what came in, what went out, why, under which instructions, and whether anybody checked it?\n\nIf yes, you can operate this automation. If no, you are not running it; it is running, and you are watching."},"check":{"question":"An automation's log records the input, the output and the timestamp. A customer disputes a reply from six weeks ago; re-running the same input today produces a different, correct-looking answer. What is the most useful thing the missing fields would have told the owner?","options":[{"text":"Whether the customer's message had been altered before it reached the model, which would explain why the stored input no longer produces the stored output.","correct":false,"tempting":"Invents tampering to explain variation that the model produces by itself, run to run."},{"text":"How long the run took and what it cost, which together indicate whether the provider silently switched to a smaller model that week.","correct":false,"tempting":"Cost and duration are genuinely useful hints about a switch, but they are indirect evidence where the model name is direct."},{"text":"Whether a person approved the reply before it was sent, which determines whether the business or the automation is answerable for what the customer received.","correct":false,"tempting":"The approval flag matters for responsibility, but the owner's question here is why the output differs."},{"text":"Which prompt version and which model produced the original, since today's re-run uses neither.","correct":true}]},"prev":{"slug":"the-duplicate-problem","title":"The duplicate problem, and the one line that fixes it","url":"https://addaly.com/learn/ai-for-small-business/the-duplicate-problem"},"next":{"slug":"the-approval-queue","title":"Designing the four-second approval","url":"https://addaly.com/learn/ai-for-small-business/the-approval-queue"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'A log that answers \"why did it do that?\" three weeks later', in AI for a Small Business. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-for-small-business/a-log-you-can-read (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"A log that answers “why did it do that?” three weeks later","courseTitle":"AI for a Small Business","module":{"index":7,"slug":"letting-it-act","title":"Letting it act: automation you can trust with the keys"},"answer":null,"keyIdea":"A run is only defensible if you kept the input, the output, the decision and the version of the prompt and model that produced it, because a model’s output is not reproducible from the prompt alone.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-question-you-will-eventually-be-asked","runs":[{"text":"The question you will eventually be asked"}]},{"t":"p","runs":[{"text":"A customer forwards you a message your automation sent five weeks ago and asks why they were told that."}]},{"t":"p","runs":[{"text":"You open the automation. The prompt has been edited twice since. The provider has upgraded the model underneath you. You re-run it with the same input and get something different and perfectly reasonable."}]},{"t":"p","runs":[{"text":"You now cannot answer the question. Not because you were careless with the automation, but because you did not keep a record, and a model’s output is not recoverable from the prompt: same input, same instructions, different day, different words."}]},{"t":"h2","id":"what-to-record-per-run","runs":[{"text":"What to record, per run"}]},{"t":"p","runs":[{"text":"Six things, and none of them is expensive:"}]},{"t":"ul","items":[[{"text":"When","b":true},{"text":", with a time zone."}],[{"text":"What went in","b":true},{"text":": the actual customer message or document, not a summary of it."}],[{"text":"What came out","b":true},{"text":": the full text or the full set of fields."}],[{"text":"What was decided","b":true},{"text":": which branch it took, what it sent, to whom, and whether a person approved."}],[{"text":"Which prompt version","b":true},{"text":": a version number you increment whenever you edit it, and the prompt text stored against that number."}],[{"text":"Which model","b":true},{"text":", by its exact name including the version suffix."}]]},{"t":"p","runs":[{"text":"Add cost and duration if it is free to do so; both are useful later for spotting a change nobody announced."}]},{"t":"h2","id":"where-it-can-live","runs":[{"text":"Where it can live"}]},{"t":"p","runs":[{"text":"A spreadsheet. Genuinely. A row per run, one tab per automation, on a business that does a few hundred runs a month. Every automation platform can append a row to a sheet, and a sheet is searchable, sortable and readable by everybody in the business without a login to anything technical."}]},{"t":"p","runs":[{"text":"Beyond a few thousand rows a month, a proper database is easier, but almost no small business reaches that with AI automations, and the sheet you will actually keep beats the system you would have to maintain."}]},{"t":"p","runs":[{"text":"Free path throughout: Google Sheets with Apps Script, LibreOffice with a CSV appended by a script, or a plain text file with one JSON object per line. The last is what most logging systems are underneath."}]},{"t":"h2","id":"why-the-prompt-version-is-the-field-people-skip","runs":[{"text":"Why the prompt version is the field people skip"}]},{"t":"p","runs":[{"text":"Because it feels redundant. The prompt is right there in the tool."}]},{"t":"p","runs":[{"text":"It is right there in its "},{"text":"current","i":true},{"text":" state. Small-business prompts get edited constantly: a customer complains about a phrase, somebody adds a rule, the price changes. Without a version stamp, every run before the last edit is unexplainable, and the one thing you most want to know when something goes wrong is whether it was caused by the edit you made on Tuesday."}]},{"t":"p","runs":[{"text":"Increment a number in the prompt itself. "},{"text":"v7","code":true},{"text":". Log the number. Keep the old text in the same file. This is the cheapest useful discipline in the whole module."}]},{"t":"h2","id":"redaction-briefly","runs":[{"text":"Redaction, briefly"}]},{"t":"p","runs":[{"text":"Your log now contains customer messages, which means it is customer data, which means module nine’s obligations apply to it as much as to your CRM. Two practical consequences: do not keep it forever (ninety days covers almost every question you will actually be asked) and do not put it somewhere with looser access than the system the data came from."}]},{"t":"p","runs":[{"text":"If the inputs contain anything sensitive, log a reference to the record rather than the content itself, and accept that you have traded some diagnostic power for a smaller problem."}]},{"t":"h2","id":"what-you-will-use-it-for-apart-from-complaints","runs":[{"text":"What you will use it for, apart from complaints"}]},{"t":"p","runs":[{"text":"Spotting a silent change.","b":true},{"text":" Average output length drops by a third one Wednesday. Nobody told you the model was updated; the log did."}]},{"t":"p","runs":[{"text":"Finding the real error rate.","b":true},{"text":" Sample twenty logged runs a month and read them properly. This is how you discover the automation has been quietly mishandling one category since May."}]},{"t":"p","runs":[{"text":"Deciding whether to climb the ladder.","b":true},{"text":" The evidence from the first lesson in this module (fifty consecutive clean items) is a claim about your log. Without one, it is a feeling."}]},{"t":"h2","id":"the-test","runs":[{"text":"The test"}]},{"t":"p","runs":[{"text":"Pick a run from a month ago at random. Can you say what came in, what went out, why, under which instructions, and whether anybody checked it?"}]},{"t":"p","runs":[{"text":"If yes, you can operate this automation. If no, you are not running it; it is running, and you are watching."}]}],"check":{"question":[{"text":"An automation’s log records the input, the output and the timestamp. A customer disputes a reply from six weeks ago; re-running the same input today produces a different, correct-looking answer. What is the most useful thing the missing fields would have told the owner?"}],"options":[{"text":[{"text":"Whether the customer’s message had been altered before it reached the model, which would explain why the stored input no longer produces the stored output."}],"correct":false,"tempting":[{"text":"Invents tampering to explain variation that the model produces by itself, run to run."}]},{"text":[{"text":"How long the run took and what it cost, which together indicate whether the provider silently switched to a smaller model that week."}],"correct":false,"tempting":[{"text":"Cost and duration are genuinely useful hints about a switch, but they are indirect evidence where the model name is direct."}]},{"text":[{"text":"Whether a person approved the reply before it was sent, which determines whether the business or the automation is answerable for what the customer received."}],"correct":false,"tempting":[{"text":"The approval flag matters for responsibility, but the owner’s question here is why the output differs."}]},{"text":[{"text":"Which prompt version and which model produced the original, since today’s re-run uses neither."}],"correct":true,"tempting":null}],"order":[2,3,0,1],"why":null},"prev":{"slug":"the-duplicate-problem","title":"The duplicate problem, and the one line that fixes it","minutes":8},"next":{"slug":"the-approval-queue","title":"Designing the four-second approval","minutes":8}}}