A log that answers “why did it do that?” three weeks later

The question you will eventually be asked

A customer forwards you a message your automation sent five weeks ago and asks why they were told that.

You open the automation. The prompt has been edited twice since. The provider has upgraded the model underneath you. You re-run it with the same input and get something different and perfectly reasonable.

You now cannot answer the question. Not because you were careless with the automation, but because you did not keep a record, and a model’s output is not recoverable from the prompt: same input, same instructions, different day, different words.

What to record, per run

Six things, and none of them is expensive:

• When, with a time zone.
• What went in: the actual customer message or document, not a summary of it.
• What came out: the full text or the full set of fields.
• What was decided: which branch it took, what it sent, to whom, and whether a person approved.
• Which prompt version: a version number you increment whenever you edit it, and the prompt text stored against that number.
• Which model, by its exact name including the version suffix.

Add cost and duration if it is free to do so; both are useful later for spotting a change nobody announced.

Where it can live

A spreadsheet. Genuinely. A row per run, one tab per automation, on a business that does a few hundred runs a month. Every automation platform can append a row to a sheet, and a sheet is searchable, sortable and readable by everybody in the business without a login to anything technical.

Beyond a few thousand rows a month, a proper database is easier, but almost no small business reaches that with AI automations, and the sheet you will actually keep beats the system you would have to maintain.

Free path throughout: Google Sheets with Apps Script, LibreOffice with a CSV appended by a script, or a plain text file with one JSON object per line. The last is what most logging systems are underneath.

Why the prompt version is the field people skip

Because it feels redundant. The prompt is right there in the tool.

It is right there in its current state. Small-business prompts get edited constantly: a customer complains about a phrase, somebody adds a rule, the price changes. Without a version stamp, every run before the last edit is unexplainable, and the one thing you most want to know when something goes wrong is whether it was caused by the edit you made on Tuesday.

Increment a number in the prompt itself. v7. Log the number. Keep the old text in the same file. This is the cheapest useful discipline in the whole module.

Redaction, briefly

Your log now contains customer messages, which means it is customer data, which means module nine’s obligations apply to it as much as to your CRM. Two practical consequences: do not keep it forever (ninety days covers almost every question you will actually be asked) and do not put it somewhere with looser access than the system the data came from.

If the inputs contain anything sensitive, log a reference to the record rather than the content itself, and accept that you have traded some diagnostic power for a smaller problem.

What you will use it for, apart from complaints

Spotting a silent change. Average output length drops by a third one Wednesday. Nobody told you the model was updated; the log did.

Finding the real error rate. Sample twenty logged runs a month and read them properly. This is how you discover the automation has been quietly mishandling one category since May.

Deciding whether to climb the ladder. The evidence from the first lesson in this module (fifty consecutive clean items) is a claim about your log. Without one, it is a feeling.

The test

Pick a run from a month ago at random. Can you say what came in, what went out, why, under which instructions, and whether anybody checked it?

If yes, you can operate this automation. If no, you are not running it; it is running, and you are watching.