{"course":{"slug":"prompting","title":"Getting Real Work out of a Model","url":"https://addaly.com/learn/prompting","level":"beginner","track":"start"},"lesson":{"slug":"prompts-that-break-later","title":"Why a prompt stops working","url":"https://addaly.com/learn/prompting/prompts-that-break-later","position":51,"of":72,"minutes":8,"answer":"When a stable prompt degrades, check the model version, the product, the input shape and the age of any hard-coded fact before rewriting.","keyIdea":"When a stable prompt degrades, check the model version, the product, the input shape and the age of any hard-coded fact before rewriting, and prefer prompts that describe a good answer over prompts that work around a behaviour."},"body":{"format":"markdown","text":"## It worked in March\n\nA prompt that has been fine for months starts producing worse output. Nothing was changed. This is common, and there are five causes worth checking in order, because four of them are not the prompt.\n\n## 1. The model changed under you\n\nProviders update models continuously. Sometimes the name changes and you can see it; sometimes the same name points at a new build. Behaviour shifts: verbosity, formatting habits, how strictly instructions are followed, where refusal boundaries sit.\n\nA prompt tuned tightly around one model's habits is the most fragile kind. If your prompt contains a line that exists to work around something (a repeated instruction because it kept forgetting, a strange phrasing that happened to help) that line is a candidate for the first thing to break.\n\n**What to do:** pin a model version where the product lets you. Record which model your prompt was tested on. Re-run the ten items after any announced update.\n\n## 2. The product changed\n\nThe system prompt, the tools, the memory behaviour, whether files are retrieved or supplied in full. These change more often than models and are announced less.\n\nA prompt that relied on a web search happening automatically will fail quietly when routing changes and the search stops firing.\n\n## 3. Your inputs changed shape\n\nThe supplier redesigned their invoice. The export added a column. Somebody started sending PDFs where they used to send text. The prompt is unchanged and the thing it operates on is not.\n\nThis is the most common cause in practice and the easiest to miss, because you look at the prompt rather than the input.\n\n**What to do:** when quality drops, look at three recent inputs beside three old ones before you touch anything.\n\n## 4. The world moved past your context\n\nA prompt with hard-coded facts (this year's prices, the current policy, a rate, a deadline) becomes confidently wrong on a date nobody marked. It does not fail. It succeeds at applying stale information.\n\n**What to do:** keep facts out of the prompt where you can, and put them in the pasted material instead. Where a fact must live in the prompt, date it in a comment: `# VAT rate as at 2026-04, check annually`.\n\n## 5. You changed it and forgot\n\nCommon, and the reason for the change log in an earlier lesson.\n\n## The brittleness audit\n\nYou can predict fragility before it bites. Read your prompt and mark every line that is:\n\n- **A workaround** for a behaviour rather than a statement of what you want.\n- **A hard-coded fact** that has a shelf life.\n- **A reference to product behaviour** (\"search the web for\", \"use the attached file\") that depends on features being enabled.\n- **A magic phrase** with no mechanism you can explain.\n\nThose lines are your breakage surface. Reducing it is straightforward: state what you want rather than how to behave, move facts into the material, and delete anything you cannot justify.\n\nThe general principle: **a prompt that says what a good answer looks like ages well; a prompt that says how to behave ages badly**, because the behaviour it compensates for is exactly what updates change.\n\n## Should you upgrade?\n\nUsually yes, and re-test rather than assuming. Newer models generally need less scaffolding, so an upgrade is a good moment to try deleting things: the three examples, the repeated instruction, the elaborate reasoning scaffold. Often two of them are no longer earning their space, and you get a shorter, cheaper, faster prompt out of the exercise.\n\nOccasionally an upgrade is worse for your specific task. That is real, it is not usually admitted, and it is exactly what your ten test items are for.\n\n## The version nobody records\n\nOne more habit, and it is the cheapest insurance in this module. When you finish tuning a prompt, write down the model and the date beside it.\n\nSix months later, \"it used to work\" is not a diagnosis anybody can act on. \"It scored 9/10 on 4 September against this model, and 5/10 today\" is a report you can take to a colleague, a vendor, or your own next hour. The difference between those two sentences is fifteen seconds of typing at a moment when you feel finished and least inclined to type anything."},"check":{"question":"A monthly extraction prompt has worked for a year and now returns \"not stated\" for the supplier VAT number on most documents. Nothing in the prompt changed. What should be checked first?","options":[{"text":"Whether the model has become more cautious and needs firmer instructions.","correct":false,"tempting":"Speculates about model disposition before looking at the cheapest and most likely explanation."},{"text":"Whether the escape hatch is now firing too readily and should be removed.","correct":false,"tempting":"Removing the escape hatch would replace visible absences with invented numbers, making a detection problem into a data-integrity one."},{"text":"Whether the context window is being exceeded by larger documents.","correct":false,"tempting":"Would produce truncation and missing rows rather than one specific field consistently absent."},{"text":"Whether the incoming documents changed shape: a new template, a different export, or scans instead of text.","correct":true}]},"prev":{"slug":"your-own-prompt-file","title":"The file that makes this compound","url":"https://addaly.com/learn/prompting/your-own-prompt-file"},"next":{"slug":"borrowed-prompts","title":"Why the prompt from the internet does not work","url":"https://addaly.com/learn/prompting/borrowed-prompts"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Why a prompt stops working', in Getting Real Work out of a Model. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/prompting/prompts-that-break-later (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Why a prompt stops working","courseTitle":"Getting Real Work out of a Model","module":{"index":6,"slug":"from-lucky-to-reliable","title":"From lucky to reliable"},"answer":null,"keyIdea":"When a stable prompt degrades, check the model version, the product, the input shape and the age of any hard-coded fact before rewriting, and prefer prompts that describe a good answer over prompts that work around a behaviour.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"it-worked-in-march","runs":[{"text":"It worked in March"}]},{"t":"p","runs":[{"text":"A prompt that has been fine for months starts producing worse output. Nothing was changed. This is common, and there are five causes worth checking in order, because four of them are not the prompt."}]},{"t":"h2","id":"1-the-model-changed-under-you","runs":[{"text":"1. The model changed under you"}]},{"t":"p","runs":[{"text":"Providers update models continuously. Sometimes the name changes and you can see it; sometimes the same name points at a new build. Behaviour shifts: verbosity, formatting habits, how strictly instructions are followed, where refusal boundaries sit."}]},{"t":"p","runs":[{"text":"A prompt tuned tightly around one model’s habits is the most fragile kind. If your prompt contains a line that exists to work around something (a repeated instruction because it kept forgetting, a strange phrasing that happened to help) that line is a candidate for the first thing to break."}]},{"t":"p","runs":[{"text":"What to do:","b":true},{"text":" pin a model version where the product lets you. Record which model your prompt was tested on. Re-run the ten items after any announced update."}]},{"t":"h2","id":"2-the-product-changed","runs":[{"text":"2. The product changed"}]},{"t":"p","runs":[{"text":"The system prompt, the tools, the memory behaviour, whether files are retrieved or supplied in full. These change more often than models and are announced less."}]},{"t":"p","runs":[{"text":"A prompt that relied on a web search happening automatically will fail quietly when routing changes and the search stops firing."}]},{"t":"h2","id":"3-your-inputs-changed-shape","runs":[{"text":"3. Your inputs changed shape"}]},{"t":"p","runs":[{"text":"The supplier redesigned their invoice. The export added a column. Somebody started sending PDFs where they used to send text. The prompt is unchanged and the thing it operates on is not."}]},{"t":"p","runs":[{"text":"This is the most common cause in practice and the easiest to miss, because you look at the prompt rather than the input."}]},{"t":"p","runs":[{"text":"What to do:","b":true},{"text":" when quality drops, look at three recent inputs beside three old ones before you touch anything."}]},{"t":"h2","id":"4-the-world-moved-past-your-context","runs":[{"text":"4. The world moved past your context"}]},{"t":"p","runs":[{"text":"A prompt with hard-coded facts (this year’s prices, the current policy, a rate, a deadline) becomes confidently wrong on a date nobody marked. It does not fail. It succeeds at applying stale information."}]},{"t":"p","runs":[{"text":"What to do:","b":true},{"text":" keep facts out of the prompt where you can, and put them in the pasted material instead. Where a fact must live in the prompt, date it in a comment: "},{"text":"# VAT rate as at 2026-04, check annually","code":true},{"text":"."}]},{"t":"h2","id":"5-you-changed-it-and-forgot","runs":[{"text":"5. You changed it and forgot"}]},{"t":"p","runs":[{"text":"Common, and the reason for the change log in an earlier lesson."}]},{"t":"h2","id":"the-brittleness-audit","runs":[{"text":"The brittleness audit"}]},{"t":"p","runs":[{"text":"You can predict fragility before it bites. Read your prompt and mark every line that is:"}]},{"t":"ul","items":[[{"text":"A workaround","b":true},{"text":" for a behaviour rather than a statement of what you want."}],[{"text":"A hard-coded fact","b":true},{"text":" that has a shelf life."}],[{"text":"A reference to product behaviour","b":true},{"text":" (“search the web for”, “use the attached file”) that depends on features being enabled."}],[{"text":"A magic phrase","b":true},{"text":" with no mechanism you can explain."}]]},{"t":"p","runs":[{"text":"Those lines are your breakage surface. Reducing it is straightforward: state what you want rather than how to behave, move facts into the material, and delete anything you cannot justify."}]},{"t":"p","runs":[{"text":"The general principle: "},{"text":"a prompt that says what a good answer looks like ages well; a prompt that says how to behave ages badly","b":true},{"text":", because the behaviour it compensates for is exactly what updates change."}]},{"t":"h2","id":"should-you-upgrade","runs":[{"text":"Should you upgrade?"}]},{"t":"p","runs":[{"text":"Usually yes, and re-test rather than assuming. Newer models generally need less scaffolding, so an upgrade is a good moment to try deleting things: the three examples, the repeated instruction, the elaborate reasoning scaffold. Often two of them are no longer earning their space, and you get a shorter, cheaper, faster prompt out of the exercise."}]},{"t":"p","runs":[{"text":"Occasionally an upgrade is worse for your specific task. That is real, it is not usually admitted, and it is exactly what your ten test items are for."}]},{"t":"h2","id":"the-version-nobody-records","runs":[{"text":"The version nobody records"}]},{"t":"p","runs":[{"text":"One more habit, and it is the cheapest insurance in this module. When you finish tuning a prompt, write down the model and the date beside it."}]},{"t":"p","runs":[{"text":"Six months later, “it used to work” is not a diagnosis anybody can act on. “It scored 9/10 on 4 September against this model, and 5/10 today” is a report you can take to a colleague, a vendor, or your own next hour. The difference between those two sentences is fifteen seconds of typing at a moment when you feel finished and least inclined to type anything."}]}],"check":{"question":[{"text":"A monthly extraction prompt has worked for a year and now returns “not stated” for the supplier VAT number on most documents. Nothing in the prompt changed. What should be checked first?"}],"options":[{"text":[{"text":"Whether the model has become more cautious and needs firmer instructions."}],"correct":false,"tempting":[{"text":"Speculates about model disposition before looking at the cheapest and most likely explanation."}]},{"text":[{"text":"Whether the escape hatch is now firing too readily and should be removed."}],"correct":false,"tempting":[{"text":"Removing the escape hatch would replace visible absences with invented numbers, making a detection problem into a data-integrity one."}]},{"text":[{"text":"Whether the context window is being exceeded by larger documents."}],"correct":false,"tempting":[{"text":"Would produce truncation and missing rows rather than one specific field consistently absent."}]},{"text":[{"text":"Whether the incoming documents changed shape: a new template, a different export, or scans instead of text."}],"correct":true,"tempting":null}],"order":[0,3,1,2],"why":null},"prev":{"slug":"your-own-prompt-file","title":"The file that makes this compound","minutes":8},"next":{"slug":"borrowed-prompts","title":"Why the prompt from the internet does not work","minutes":8}}}