{"course":{"slug":"ai-for-small-business","title":"AI for a Small Business","url":"https://addaly.com/learn/ai-for-small-business","level":"beginner","track":"work"},"lesson":{"slug":"keeping-the-knowledge-current","title":"The stale document problem","url":"https://addaly.com/learn/ai-for-small-business/keeping-the-knowledge-current","position":27,"of":88,"minutes":8,"answer":"A grounded system inherits the currency of its documents exactly, so an out-of-date source produces confident cited errors indefinitely, which is why every document carries a date and a named owner.","keyIdea":"A grounded system inherits the currency of its documents exactly, so an out-of-date source produces confident cited errors indefinitely, which is why every document carries a date and a named owner, superseded versions leave the library, and a change to a price is a change to the facts sheet on the same checklist."},"body":{"format":"markdown","text":"## The failure that looks like success\n\nA grounded assistant is accurate for four months. Everyone stops checking, reasonably, because it has earned it.\n\nThen the delivery charge changes and the document does not. From that day, every answer about delivery is wrong: fluently, with a citation, in the house voice. There is no error message, no hesitation, no drop in quality of any kind that a reader could notice. The system has become a machine for repeating an outdated fact at scale.\n\nThis is the most common way a working AI setup turns into a liability, and it is entirely a process problem rather than a technical one.\n\n## Where staleness comes from\n\n**Nobody owns the document.** It was written once, by whoever set the system up, and it belongs to no one.\n\n**The change happened somewhere else.** You updated the price in the shop, on the menu and in the till. The facts sheet was the fourth place and nobody thought of it.\n\n**Superseded versions were left in the folder.** `terms-v2.pdf` and `terms-final.pdf` both sit in the library, and retrieval has no idea which is current. This is the one that produces contradictory answers depending on how the question is phrased.\n\n**The source was never true.** A procedure describing how things worked before the shop moved. Nobody noticed, because nobody reads their own procedures.\n\n## Four controls\n\n**Date every document, at the top, in the text.** Not the file's modified date, which changes when somebody opens it. A line in the body: `Current as of 12 August 2026.` The model can then be told to say when its source was last updated, which surfaces staleness in the answer itself.\n\n**One owner per document, named in it.** A person, not a role. In a business of four people this feels excessive and is the reason it gets done at all.\n\n**Delete, do not archive in place.** Superseded documents leave the library. Keep them somewhere else if you need the history. A file labelled `OLD` is not labelled at all as far as retrieval is concerned.\n\n**Put a change on your existing checklist.** When prices change, the facts sheet is on the same list as the menu and the website. This is the control that actually works, because it attaches the task to something that already happens.\n\n## The quarterly twenty minutes\n\nOnce a quarter, take your ten most-asked questions and run them. Read the answers as a customer would, and check each against reality.\n\nYou will typically find one wrong answer per quarter in an active business. That is not a sign of a bad system; it is what a business that changes things looks like. Finding it in a twenty-minute review is a great deal cheaper than finding it when a customer holds you to it.\n\nKeep a note of what you found and when. Six months of that note tells you how fast your material goes stale, which tells you whether quarterly is often enough. A shop with fixed prices might manage twice a year. A business whose costs move with an exchange rate needs it monthly.\n\n## When something changes today\n\nTwo things, in this order. Fix the document. Then run the two or three questions that touch it, to confirm the answers changed.\n\nThe second step catches the case where the tool has cached the old version, which some do, or where a superseded copy is still being retrieved in preference to the one you just edited. Both are common enough to be worth the sixty seconds, and both are invisible if you only check that the file looks right.\n\n## The other kind of staleness\n\nDocuments go out of date. So does the world the documents describe, in ways no edit will fix.\n\nA procedure written when you had two staff assumes two staff. A price list built around a supplier you no longer use still has their part numbers. A frequently-asked-questions page answers the questions people asked in 2024, and the questions have changed because your customers have.\n\nThis is why the quarterly review starts with the ten most-asked questions rather than with the documents. Questions reveal drift that reading your own material never will, because you read your own material remembering what it was meant to say.\n\n## Who should do the review\n\nNot the person who wrote the documents. They will read what they intended.\n\nThe best reviewer is whoever answers customers most: they know what is actually being asked and they will notice immediately when an answer does not match what they say on the phone. In a business of two, swap: each person reviews the other's material. In a business of one, leave a week between writing and reviewing, and read the answers on a phone rather than a computer. Both tricks work by making the material slightly unfamiliar, which is the whole trick."},"check":{"question":"After updating its price list document, a business notices the assistant still quotes the old figure for one product. What should it do first?","options":[{"text":"Rebuild the whole document library from scratch, since a partially updated index cannot be trusted for any answer.","correct":false,"tempting":"Escalates to the most expensive remedy before establishing what happened, and would leave the same superseded file in the folder to cause the same problem again."},{"text":"Re-run the two or three questions that touch the changed prices, to check whether the tool has cached the old version or is retrieving a superseded copy.","correct":true},{"text":"Add an instruction telling the model to prefer the most recently modified document, since the file dates make the correct version identifiable.","correct":false,"tempting":"Relies on modification dates the retrieval step usually does not see, and dates change whenever a file is merely opened rather than edited."},{"text":"Accept it as a known limitation and tell staff to verify prices independently before quoting them to a customer, since no document system can be kept perfectly current in a business where prices move frequently.","correct":false,"tempting":"Converts a fixable indexing problem into a permanent manual workaround, which removes most of the reason for having a grounded assistant at all."}]},"prev":{"slug":"meetings-calls-and-notes","title":"Transcription, and the consent question nobody asks","url":"https://addaly.com/learn/ai-for-small-business/meetings-calls-and-notes"},"next":{"slug":"when-the-answer-is-not-in-the-file","title":"Checking a grounded answer in under a minute","url":"https://addaly.com/learn/ai-for-small-business/when-the-answer-is-not-in-the-file"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The stale document problem', in AI for a Small Business. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-for-small-business/keeping-the-knowledge-current (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The stale document problem","courseTitle":"AI for a Small Business","module":{"index":3,"slug":"your-own-knowledge","title":"Your own documents, and getting straight answers out of them"},"answer":null,"keyIdea":"A grounded system inherits the currency of its documents exactly, so an out-of-date source produces confident cited errors indefinitely, which is why every document carries a date and a named owner, superseded versions leave the library, and a change to a price is a change to the facts sheet on the same checklist.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-failure-that-looks-like-success","runs":[{"text":"The failure that looks like success"}]},{"t":"p","runs":[{"text":"A grounded assistant is accurate for four months. Everyone stops checking, reasonably, because it has earned it."}]},{"t":"p","runs":[{"text":"Then the delivery charge changes and the document does not. From that day, every answer about delivery is wrong: fluently, with a citation, in the house voice. There is no error message, no hesitation, no drop in quality of any kind that a reader could notice. The system has become a machine for repeating an outdated fact at scale."}]},{"t":"p","runs":[{"text":"This is the most common way a working AI setup turns into a liability, and it is entirely a process problem rather than a technical one."}]},{"t":"h2","id":"where-staleness-comes-from","runs":[{"text":"Where staleness comes from"}]},{"t":"p","runs":[{"text":"Nobody owns the document.","b":true},{"text":" It was written once, by whoever set the system up, and it belongs to no one."}]},{"t":"p","runs":[{"text":"The change happened somewhere else.","b":true},{"text":" You updated the price in the shop, on the menu and in the till. The facts sheet was the fourth place and nobody thought of it."}]},{"t":"p","runs":[{"text":"Superseded versions were left in the folder.","b":true},{"text":" "},{"text":"terms-v2.pdf","code":true},{"text":" and "},{"text":"terms-final.pdf","code":true},{"text":" both sit in the library, and retrieval has no idea which is current. This is the one that produces contradictory answers depending on how the question is phrased."}]},{"t":"p","runs":[{"text":"The source was never true.","b":true},{"text":" A procedure describing how things worked before the shop moved. Nobody noticed, because nobody reads their own procedures."}]},{"t":"h2","id":"four-controls","runs":[{"text":"Four controls"}]},{"t":"p","runs":[{"text":"Date every document, at the top, in the text.","b":true},{"text":" Not the file’s modified date, which changes when somebody opens it. A line in the body: "},{"text":"Current as of 12 August 2026.","code":true},{"text":" The model can then be told to say when its source was last updated, which surfaces staleness in the answer itself."}]},{"t":"p","runs":[{"text":"One owner per document, named in it.","b":true},{"text":" A person, not a role. In a business of four people this feels excessive and is the reason it gets done at all."}]},{"t":"p","runs":[{"text":"Delete, do not archive in place.","b":true},{"text":" Superseded documents leave the library. Keep them somewhere else if you need the history. A file labelled "},{"text":"OLD","code":true},{"text":" is not labelled at all as far as retrieval is concerned."}]},{"t":"p","runs":[{"text":"Put a change on your existing checklist.","b":true},{"text":" When prices change, the facts sheet is on the same list as the menu and the website. This is the control that actually works, because it attaches the task to something that already happens."}]},{"t":"h2","id":"the-quarterly-twenty-minutes","runs":[{"text":"The quarterly twenty minutes"}]},{"t":"p","runs":[{"text":"Once a quarter, take your ten most-asked questions and run them. Read the answers as a customer would, and check each against reality."}]},{"t":"p","runs":[{"text":"You will typically find one wrong answer per quarter in an active business. That is not a sign of a bad system; it is what a business that changes things looks like. Finding it in a twenty-minute review is a great deal cheaper than finding it when a customer holds you to it."}]},{"t":"p","runs":[{"text":"Keep a note of what you found and when. Six months of that note tells you how fast your material goes stale, which tells you whether quarterly is often enough. A shop with fixed prices might manage twice a year. A business whose costs move with an exchange rate needs it monthly."}]},{"t":"h2","id":"when-something-changes-today","runs":[{"text":"When something changes today"}]},{"t":"p","runs":[{"text":"Two things, in this order. Fix the document. Then run the two or three questions that touch it, to confirm the answers changed."}]},{"t":"p","runs":[{"text":"The second step catches the case where the tool has cached the old version, which some do, or where a superseded copy is still being retrieved in preference to the one you just edited. Both are common enough to be worth the sixty seconds, and both are invisible if you only check that the file looks right."}]},{"t":"h2","id":"the-other-kind-of-staleness","runs":[{"text":"The other kind of staleness"}]},{"t":"p","runs":[{"text":"Documents go out of date. So does the world the documents describe, in ways no edit will fix."}]},{"t":"p","runs":[{"text":"A procedure written when you had two staff assumes two staff. A price list built around a supplier you no longer use still has their part numbers. A frequently-asked-questions page answers the questions people asked in 2024, and the questions have changed because your customers have."}]},{"t":"p","runs":[{"text":"This is why the quarterly review starts with the ten most-asked questions rather than with the documents. Questions reveal drift that reading your own material never will, because you read your own material remembering what it was meant to say."}]},{"t":"h2","id":"who-should-do-the-review","runs":[{"text":"Who should do the review"}]},{"t":"p","runs":[{"text":"Not the person who wrote the documents. They will read what they intended."}]},{"t":"p","runs":[{"text":"The best reviewer is whoever answers customers most: they know what is actually being asked and they will notice immediately when an answer does not match what they say on the phone. In a business of two, swap: each person reviews the other’s material. In a business of one, leave a week between writing and reviewing, and read the answers on a phone rather than a computer. Both tricks work by making the material slightly unfamiliar, which is the whole trick."}]}],"check":{"question":[{"text":"After updating its price list document, a business notices the assistant still quotes the old figure for one product. What should it do first?"}],"options":[{"text":[{"text":"Rebuild the whole document library from scratch, since a partially updated index cannot be trusted for any answer."}],"correct":false,"tempting":[{"text":"Escalates to the most expensive remedy before establishing what happened, and would leave the same superseded file in the folder to cause the same problem again."}]},{"text":[{"text":"Re-run the two or three questions that touch the changed prices, to check whether the tool has cached the old version or is retrieving a superseded copy."}],"correct":true,"tempting":null},{"text":[{"text":"Add an instruction telling the model to prefer the most recently modified document, since the file dates make the correct version identifiable."}],"correct":false,"tempting":[{"text":"Relies on modification dates the retrieval step usually does not see, and dates change whenever a file is merely opened rather than edited."}]},{"text":[{"text":"Accept it as a known limitation and tell staff to verify prices independently before quoting them to a customer, since no document system can be kept perfectly current in a business where prices move frequently."}],"correct":false,"tempting":[{"text":"Converts a fixable indexing problem into a permanent manual workaround, which removes most of the reason for having a grounded assistant at all."}]}],"order":[2,1,3,0],"why":null},"prev":{"slug":"meetings-calls-and-notes","title":"Transcription, and the consent question nobody asks","minutes":9},"next":{"slug":"when-the-answer-is-not-in-the-file","title":"Checking a grounded answer in under a minute","minutes":8}}}