{"course":{"slug":"prompting","title":"Getting Real Work out of a Model","url":"https://addaly.com/learn/prompting","level":"beginner","track":"start"},"lesson":{"slug":"change-one-thing","title":"Change one thing at a time","url":"https://addaly.com/learn/prompting/change-one-thing","position":48,"of":72,"minutes":8,"answer":"Change one element at a time against fixed test items with several runs each, discard what does not measurably help, and periodically try removing things: most reused prompts are a third decoration.","keyIdea":"Change one element at a time against fixed test items with several runs each, discard what does not measurably help, and periodically try removing things: most reused prompts are a third decoration."},"body":{"format":"markdown","text":"## The rewrite that teaches you nothing\n\nThe output is disappointing. You rewrite the prompt: add three examples, tighten the instruction, add a format spec, and paste more context. The new output is better.\n\nWhat helped? You have no idea, and you have no idea what hurt either: one of those four changes may have made things worse while the other three carried it.\n\nThis matters because prompts accumulate. Six months on, that prompt has twelve elements, four of them useless, two of them harmful, and nobody dares remove anything because nobody knows which is which. Every team that uses these tools seriously ends up with a prompt like that.\n\n## The discipline\n\n**One change. Same test items. Several runs. Keep or discard.**\n\nThat is the whole method, and each part matters.\n\n**One change** so the result is attributable.\n\n**The same test items**: five to ten, with known answers, held out from the prompt. Without them you are comparing today's impression to yesterday's memory.\n\n**Several runs** because of sampling. Two runs of the old version and two of the new is the bare minimum; three each is better. A change that shows up only in one run out of three has not shown up.\n\n**Keep or discard** explicitly. A change that did not help comes back out. This is the step everybody skips, and it is why prompts get long.\n\n## What counts as one change\n\nAdding examples is one change. Adding examples *and* rewording the task is two.\n\nRewording the task is one change. Rewording it and adding a format specification is two.\n\nThe unit is a hypothesis: *I think the failures are caused by X, so I am doing Y.* If you cannot state that sentence about a change, you are decorating rather than iterating.\n\n## Removal is a change too\n\nOnce a prompt is working, run the experiment backwards. Take out the oldest element nobody remembers adding. Run the test. If the score does not move, it stays out.\n\nThis is more valuable than it sounds. A shorter prompt is cheaper, faster, easier to read, and less likely to contain a constraint quietly fighting another one. Most inherited prompts can lose a third of their length with no measurable loss, and the exercise usually surfaces one instruction that was actively harmful.\n\n## When you cannot be this rigorous\n\nMost of the time you cannot, and should not try. For a one-off task in a chat window, iterate however you like: you will read the answer, so the feedback is immediate and complete.\n\nThe discipline is for prompts that will be **reused**: run over a batch, used weekly, shared with colleagues, or embedded in something. That is where an unattributed improvement becomes a permanent mystery.\n\nThe threshold is roughly: *will anybody, including me, run this again without reading every output?* If yes, do it properly. It costs half an hour once.\n\n## Keep the record\n\nThree lines per change, in the same file as the prompt:\n\n```\n2026-09-02  Added 3 boundary examples (refund/delivery edge)\n            8/10 → 9/10 over 3 runs. Kept.\n2026-09-04  Added \"be thorough and precise\" to the opening.\n            9/10 → 9/10. Removed.\n2026-09-11  Required a quoted sentence per extracted value.\n            9/10 → 10/10, and the failures became visible. Kept.\n```\n\nThat record is worth more than the prompt. It stops you re-trying things that did not work, it tells a colleague why the prompt looks the way it does, and on the day a model update changes everything it tells you what to re-test first.\n\n> Figure: A change log that earns its keep. Timeline: 2 Sep, Added three boundary examples on the refund/delivery edge. 8/10 to 9/10 over three runs. Kept.; 4 Sep, Added 'be thorough and precise' to the opening. 9/10 to 9/10. Removed.; 11 Sep, Required a quoted sentence per extracted value. 9/10 to 10/10, and the failures became visible. Kept.; 3 Oct, Provider announced a model update. Re-ran the ten items before touching anything. Still 10/10.. Three lines per change, beside the prompt. It stops you re-trying what failed, tells a colleague why the prompt looks the way it does, and on the day a model update changes everything it says what to re-test first.\n\n## The order to try things in\n\nWhen several changes are candidates, try them in order of expected payoff, which is roughly the order of the four causes from the last lesson.\n\n1. **Add missing material.** Biggest effect, almost always.\n2. **Sharpen the task**: verb, object, finished state.\n3. **Add a fallback** for the case the prompt does not cover. Small change, and it converts invisible failures into visible ones, which changes what you can measure next.\n4. **Add or change examples.** Real effect, real cost in space.\n5. **Change the format.** Usually about usability rather than accuracy.\n6. **Adjust phrasing.** Last, and honestly, mostly not worth the attempt.\n\nMost people start at six and work upwards, which is why iteration feels unrewarding to them. Starting at one is the same amount of typing and a different experience."},"check":{"question":"A developer improves a prompt by making four changes at once; accuracy on the test set rises from 6/10 to 8/10. What is the problem with stopping there?","options":[{"text":"Nothing: the goal is a better prompt, and the prompt is better.","correct":false,"tempting":"True for today and false for the next six months; an unattributed improvement cannot be maintained, reduced or diagnosed."},{"text":"Two extra correct items is within sampling noise and probably not real.","correct":false,"tempting":"Worth checking with more runs, but the deeper issue stands even if the improvement is genuine."},{"text":"The test set is too small to justify keeping any of the changes.","correct":false,"tempting":"Ten items can detect a two-item shift adequately; the flaw is the bundling, not the sample size."},{"text":"One or more of the four changes may be harmful or useless, and with no attribution the prompt can only grow from here.","correct":true}]},"prev":{"slug":"diagnosing-a-bad-answer","title":"Four reasons an answer is bad","url":"https://addaly.com/learn/prompting/diagnosing-a-bad-answer"},"next":{"slug":"the-regression-check","title":"The five items you keep forever","url":"https://addaly.com/learn/prompting/the-regression-check"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Change one thing at a time', in Getting Real Work out of a Model. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/prompting/change-one-thing (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Change one thing at a time","courseTitle":"Getting Real Work out of a Model","module":{"index":6,"slug":"from-lucky-to-reliable","title":"From lucky to reliable"},"answer":null,"keyIdea":"Change one element at a time against fixed test items with several runs each, discard what does not measurably help, and periodically try removing things: most reused prompts are a third decoration.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"timeline","title":"A change log that earns its keep","events":[{"when":"2 Sep","what":"Added three boundary examples on the refund/delivery edge. 8/10 to 9/10 over three runs. Kept."},{"when":"4 Sep","what":"Added ‘be thorough and precise’ to the opening. 9/10 to 9/10. Removed."},{"when":"11 Sep","what":"Required a quoted sentence per extracted value. 9/10 to 10/10, and the failures became visible. Kept."},{"when":"3 Oct","what":"Provider announced a model update. Re-ran the ten items before touching anything. Still 10/10."}],"caption":"Three lines per change, beside the prompt. It stops you re-trying what failed, tells a colleague why the prompt looks the way it does, and on the day a model update changes everything it says what to re-test first."},"alt":"Figure: A change log that earns its keep. Timeline: 2 Sep, Added three boundary examples on the refund/delivery edge. 8/10 to 9/10 over three runs. Kept.; 4 Sep, Added ‘be thorough and precise’ to the opening. 9/10 to 9/10. Removed.; 11 Sep, Required a quoted sentence per extracted value. 9/10 to 10/10, and the failures became visible. Kept.; 3 Oct, Provider announced a model update. Re-ran the ten items before touching anything. Still 10/10.. Three lines per change, beside the prompt. It stops you re-trying what failed, tells a colleague why the prompt looks the way it does, and on the day a model update changes everything it says what to re-test first."}},"blocks":[{"t":"h2","id":"the-rewrite-that-teaches-you-nothing","runs":[{"text":"The rewrite that teaches you nothing"}]},{"t":"p","runs":[{"text":"The output is disappointing. You rewrite the prompt: add three examples, tighten the instruction, add a format spec, and paste more context. The new output is better."}]},{"t":"p","runs":[{"text":"What helped? You have no idea, and you have no idea what hurt either: one of those four changes may have made things worse while the other three carried it."}]},{"t":"p","runs":[{"text":"This matters because prompts accumulate. Six months on, that prompt has twelve elements, four of them useless, two of them harmful, and nobody dares remove anything because nobody knows which is which. Every team that uses these tools seriously ends up with a prompt like that."}]},{"t":"h2","id":"the-discipline","runs":[{"text":"The discipline"}]},{"t":"p","runs":[{"text":"One change. Same test items. Several runs. Keep or discard.","b":true}]},{"t":"p","runs":[{"text":"That is the whole method, and each part matters."}]},{"t":"p","runs":[{"text":"One change","b":true},{"text":" so the result is attributable."}]},{"t":"p","runs":[{"text":"The same test items","b":true},{"text":": five to ten, with known answers, held out from the prompt. Without them you are comparing today’s impression to yesterday’s memory."}]},{"t":"p","runs":[{"text":"Several runs","b":true},{"text":" because of sampling. Two runs of the old version and two of the new is the bare minimum; three each is better. A change that shows up only in one run out of three has not shown up."}]},{"t":"p","runs":[{"text":"Keep or discard","b":true},{"text":" explicitly. A change that did not help comes back out. This is the step everybody skips, and it is why prompts get long."}]},{"t":"h2","id":"what-counts-as-one-change","runs":[{"text":"What counts as one change"}]},{"t":"p","runs":[{"text":"Adding examples is one change. Adding examples "},{"text":"and","i":true},{"text":" rewording the task is two."}]},{"t":"p","runs":[{"text":"Rewording the task is one change. Rewording it and adding a format specification is two."}]},{"t":"p","runs":[{"text":"The unit is a hypothesis: "},{"text":"I think the failures are caused by X, so I am doing Y.","i":true},{"text":" If you cannot state that sentence about a change, you are decorating rather than iterating."}]},{"t":"h2","id":"removal-is-a-change-too","runs":[{"text":"Removal is a change too"}]},{"t":"p","runs":[{"text":"Once a prompt is working, run the experiment backwards. Take out the oldest element nobody remembers adding. Run the test. If the score does not move, it stays out."}]},{"t":"p","runs":[{"text":"This is more valuable than it sounds. A shorter prompt is cheaper, faster, easier to read, and less likely to contain a constraint quietly fighting another one. Most inherited prompts can lose a third of their length with no measurable loss, and the exercise usually surfaces one instruction that was actively harmful."}]},{"t":"h2","id":"when-you-cannot-be-this-rigorous","runs":[{"text":"When you cannot be this rigorous"}]},{"t":"p","runs":[{"text":"Most of the time you cannot, and should not try. For a one-off task in a chat window, iterate however you like: you will read the answer, so the feedback is immediate and complete."}]},{"t":"p","runs":[{"text":"The discipline is for prompts that will be "},{"text":"reused","b":true},{"text":": run over a batch, used weekly, shared with colleagues, or embedded in something. That is where an unattributed improvement becomes a permanent mystery."}]},{"t":"p","runs":[{"text":"The threshold is roughly: "},{"text":"will anybody, including me, run this again without reading every output?","i":true},{"text":" If yes, do it properly. It costs half an hour once."}]},{"t":"h2","id":"keep-the-record","runs":[{"text":"Keep the record"}]},{"t":"p","runs":[{"text":"Three lines per change, in the same file as the prompt:"}]},{"t":"code","lang":"","label":"Text","text":"2026-09-02  Added 3 boundary examples (refund/delivery edge)\n            8/10 → 9/10 over 3 runs. Kept.\n2026-09-04  Added \"be thorough and precise\" to the opening.\n            9/10 → 9/10. Removed.\n2026-09-11  Required a quoted sentence per extracted value.\n            9/10 → 10/10, and the failures became visible. Kept."},{"t":"p","runs":[{"text":"That record is worth more than the prompt. It stops you re-trying things that did not work, it tells a colleague why the prompt looks the way it does, and on the day a model update changes everything it tells you what to re-test first."}]},{"t":"h2","id":"the-order-to-try-things-in","runs":[{"text":"The order to try things in"}]},{"t":"p","runs":[{"text":"When several changes are candidates, try them in order of expected payoff, which is roughly the order of the four causes from the last lesson."}]},{"t":"ol","start":1,"items":[[{"text":"Add missing material.","b":true},{"text":" Biggest effect, almost always."}],[{"text":"Sharpen the task","b":true},{"text":": verb, object, finished state."}],[{"text":"Add a fallback","b":true},{"text":" for the case the prompt does not cover. Small change, and it converts invisible failures into visible ones, which changes what you can measure next."}],[{"text":"Add or change examples.","b":true},{"text":" Real effect, real cost in space."}],[{"text":"Change the format.","b":true},{"text":" Usually about usability rather than accuracy."}],[{"text":"Adjust phrasing.","b":true},{"text":" Last, and honestly, mostly not worth the attempt."}]]},{"t":"p","runs":[{"text":"Most people start at six and work upwards, which is why iteration feels unrewarding to them. Starting at one is the same amount of typing and a different experience."}]}],"check":{"question":[{"text":"A developer improves a prompt by making four changes at once; accuracy on the test set rises from 6/10 to 8/10. What is the problem with stopping there?"}],"options":[{"text":[{"text":"Nothing: the goal is a better prompt, and the prompt is better."}],"correct":false,"tempting":[{"text":"True for today and false for the next six months; an unattributed improvement cannot be maintained, reduced or diagnosed."}]},{"text":[{"text":"Two extra correct items is within sampling noise and probably not real."}],"correct":false,"tempting":[{"text":"Worth checking with more runs, but the deeper issue stands even if the improvement is genuine."}]},{"text":[{"text":"The test set is too small to justify keeping any of the changes."}],"correct":false,"tempting":[{"text":"Ten items can detect a two-item shift adequately; the flaw is the bundling, not the sample size."}]},{"text":[{"text":"One or more of the four changes may be harmful or useless, and with no attribution the prompt can only grow from here."}],"correct":true,"tempting":null}],"order":[0,2,3,1],"why":null},"prev":{"slug":"diagnosing-a-bad-answer","title":"Four reasons an answer is bad","minutes":9},"next":{"slug":"the-regression-check","title":"The five items you keep forever","minutes":8}}}