{"course":{"slug":"choosing-your-tools","title":"Choosing and Using the Tools","url":"https://addaly.com/learn/choosing-your-tools","level":"beginner","track":"work"},"lesson":{"slug":"the-prompt-file","title":"The file you will keep for years","url":"https://addaly.com/learn/choosing-your-tools/the-prompt-file","position":11,"of":76,"minutes":9,"answer":"Twelve real prompts in a plain text file, each with a written note of what a good answer contains, decided before you see any answers.","keyIdea":"Twelve real prompts in a plain text file, each with a written note of what a good answer contains, decided before you see any answers, is a better guide to tool choice than any published comparison."},"body":{"format":"markdown","text":"## One file, twelve prompts, no software\n\nThe single most valuable artefact in this course is a plain text file on your own disk. It holds a dozen real tasks from your own work, and it is the thing you run whenever you need to decide between tools, whenever a tool seems to have changed, and whenever somebody tells you a new model is remarkable.\n\nIt is better than every published comparison for one reason: it measures fit to *your* work. Benchmarks measure fit to a benchmark, and the two diverge more than people expect. A model that leads on graduate-level science questions may write worse Marathi business emails than one three places below it.\n\n## What goes in it\n\nTwelve prompts, drawn from what you do. Aim for this spread:\n\n1. **Two everyday drafting tasks.** A real email with the context you would give a colleague. A short piece in your own voice.\n2. **Two document tasks.** A real document plus the question you would actually ask about it. Make one of them long.\n3. **Two in your own language**, if that is not English. One casual, one formal.\n4. **Two known-answer probes.** Questions where you already know the correct answer. The next lesson is entirely about these.\n5. **One structured task.** Extract these six fields into a table, or convert this mess into a list.\n6. **One reasoning task** from your work. A scheduling constraint, a cost comparison, a piece of logic you had to think about.\n7. **One task you have failed to get done before.** The thing you gave up on.\n8. **One edge case for your field.** A medical phrasing, a legal citation, a security question, a financial term: whichever your job puts near a refusal boundary.\n\nUse real material, not made-up examples. Made-up material is always tidier than the real thing and every tool looks better on it.\n\n## The format\n\nPlain text. Not a spreadsheet, not a notes app that syncs, not a document in a tool you might stop paying for. A `.txt` or `.md` file you can open in 2032.\n\n```\n--- 03  supplier chase, Hindi ---\ncontext: distributor 6 weeks late, we have paid 40% advance,\n         want firm date without losing the relationship\nprompt:  <the actual text you would send a model>\ngood:    polite, gives a deadline, does not grovel,\n         natural Hindi not translated-English Hindi\nlast run: 2026-03-14  toolA 2  toolB 1  local-qwen 2\n```\n\nThe `good:` line matters more than it looks. Writing down what a good answer contains **before** you see any answers is what stops you rationalising afterwards. It takes two minutes per prompt and it is the difference between a test and a vibe.\n\n> Figure: What goes in the twelve-prompt file. Bars (prompts out of twelve): Everyday drafting, in your own voice 2; A real document plus the question you would ask 2; Your own language: one casual, one formal 2; Known-answer probes 2; Structured extraction into fields 1; A reasoning task from your work 1; The task you gave up on before 1; The edge case for your field 1. All of it real material, never invented examples, because invented material is tidier than the real thing and every tool looks better on it. The two known-answer probes are the ones that catch a tool which impresses you and misleads you.\n\n## Keeping it honest over time\n\n**Do not fix a prompt because a tool failed it.** That is how a test set stops testing. If the prompt is badly written, rewrite it and reset the history, noting that you did.\n\n**Do not let it become easy.** As you get better at prompting, your prompts get clearer, and clearer prompts are easier. Every six months, add one hard new item and retire nothing.\n\n**Keep the failures.** When a tool gets something wrong in an interesting way, that becomes prompt thirteen.\n\n**Redact once, properly.** These prompts will be pasted into other people's services. Replace real names, account numbers and addresses with consistent stand-ins the first time you build the file, and you never have to think about it again. \"The supplier\" works exactly as well as the company's name, and the answer is the same.\n\n## What it costs and what it saves\n\nBuilding it: one evening. Running it across three tools: about forty minutes, twice a year.\n\nAgainst that, the thing it prevents. People switch tools on the strength of a launch video, spend a fortnight rebuilding habits, and discover the new one is worse at the one thing they do most. Or they stay on a paid plan for two years after a free tier overtook it. Forty minutes twice a year is a very cheap insurance policy against both.\n\nThere is one more use, and it is the one people are most grateful for later. When you doubt yourself (*has this got worse, or have I got fussier?*), the file answers. Memory of software quality is unreliable in a specific direction: we remember the early impressive answers and forget the early failures, so everything feels like it has declined. The file is the only witness that is not your own recollection."},"check":{"question":"You write the twelve prompts and record what each answer should contain, but only after reading the first tool's replies. What has that cost you?","options":[{"text":"Nothing, as long as the same criteria are then applied identically to every other tool.","correct":false,"tempting":"Assumes fairness is only about consistency; criteria derived from one tool's output describe that tool, so applying them evenly still measures how closely rivals imitate it."},{"text":"The criteria now describe what the first tool produced, so the test measures similarity to it.","correct":true},{"text":"Only the ordering is affected, and rotating which tool goes first on the next run corrects it.","correct":false,"tempting":"Treats an anchoring problem as an order effect; rotation helps with fatigue and ordering, but the criteria themselves stay contaminated."},{"text":"The prompts are still valid but the scoring should be redone by a second person to remove your bias from the judgement.","correct":false,"tempting":"Adds a reviewer to a standard that is already shaped by the first tool's output, so a second person applies the same contaminated criteria."}]},"prev":{"slug":"real-differences","title":"Where they differ, and where it is marketing","url":"https://addaly.com/learn/choosing-your-tools/real-differences"},"next":{"slug":"known-answer-probes","title":"The prompts where you already know the answer","url":"https://addaly.com/learn/choosing-your-tools/known-answer-probes"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The file you will keep for years', in Choosing and Using the Tools. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/choosing-your-tools/the-prompt-file (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The file you will keep for years","courseTitle":"Choosing and Using the Tools","module":{"index":2,"slug":"testing-it-yourself","title":"Testing a tool yourself"},"answer":null,"keyIdea":"Twelve real prompts in a plain text file, each with a written note of what a good answer contains, decided before you see any answers, is a better guide to tool choice than any published comparison.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"bars","unit":"prompts out of twelve","title":"What goes in the twelve-prompt file","series":[{"label":"Everyday drafting, in your own voice","value":2},{"label":"A real document plus the question you would ask","value":2},{"label":"Your own language: one casual, one formal","value":2},{"label":"Known-answer probes","value":2,"highlight":true},{"label":"Structured extraction into fields","value":1},{"label":"A reasoning task from your work","value":1},{"label":"The task you gave up on before","value":1},{"label":"The edge case for your field","value":1}],"caption":"All of it real material, never invented examples, because invented material is tidier than the real thing and every tool looks better on it. The two known-answer probes are the ones that catch a tool which impresses you and misleads you."},"alt":"Figure: What goes in the twelve-prompt file. Bars (prompts out of twelve): Everyday drafting, in your own voice 2; A real document plus the question you would ask 2; Your own language: one casual, one formal 2; Known-answer probes 2; Structured extraction into fields 1; A reasoning task from your work 1; The task you gave up on before 1; The edge case for your field 1. All of it real material, never invented examples, because invented material is tidier than the real thing and every tool looks better on it. The two known-answer probes are the ones that catch a tool which impresses you and misleads you."}},"blocks":[{"t":"h2","id":"one-file-twelve-prompts-no-software","runs":[{"text":"One file, twelve prompts, no software"}]},{"t":"p","runs":[{"text":"The single most valuable artefact in this course is a plain text file on your own disk. It holds a dozen real tasks from your own work, and it is the thing you run whenever you need to decide between tools, whenever a tool seems to have changed, and whenever somebody tells you a new model is remarkable."}]},{"t":"p","runs":[{"text":"It is better than every published comparison for one reason: it measures fit to "},{"text":"your","i":true},{"text":" work. Benchmarks measure fit to a benchmark, and the two diverge more than people expect. A model that leads on graduate-level science questions may write worse Marathi business emails than one three places below it."}]},{"t":"h2","id":"what-goes-in-it","runs":[{"text":"What goes in it"}]},{"t":"p","runs":[{"text":"Twelve prompts, drawn from what you do. Aim for this spread:"}]},{"t":"ol","start":1,"items":[[{"text":"Two everyday drafting tasks.","b":true},{"text":" A real email with the context you would give a colleague. A short piece in your own voice."}],[{"text":"Two document tasks.","b":true},{"text":" A real document plus the question you would actually ask about it. Make one of them long."}],[{"text":"Two in your own language","b":true},{"text":", if that is not English. One casual, one formal."}],[{"text":"Two known-answer probes.","b":true},{"text":" Questions where you already know the correct answer. The next lesson is entirely about these."}],[{"text":"One structured task.","b":true},{"text":" Extract these six fields into a table, or convert this mess into a list."}],[{"text":"One reasoning task","b":true},{"text":" from your work. A scheduling constraint, a cost comparison, a piece of logic you had to think about."}],[{"text":"One task you have failed to get done before.","b":true},{"text":" The thing you gave up on."}],[{"text":"One edge case for your field.","b":true},{"text":" A medical phrasing, a legal citation, a security question, a financial term: whichever your job puts near a refusal boundary."}]]},{"t":"p","runs":[{"text":"Use real material, not made-up examples. Made-up material is always tidier than the real thing and every tool looks better on it."}]},{"t":"h2","id":"the-format","runs":[{"text":"The format"}]},{"t":"p","runs":[{"text":"Plain text. Not a spreadsheet, not a notes app that syncs, not a document in a tool you might stop paying for. A "},{"text":".txt","code":true},{"text":" or "},{"text":".md","code":true},{"text":" file you can open in 2032."}]},{"t":"code","lang":"","label":"Text","text":"--- 03  supplier chase, Hindi ---\ncontext: distributor 6 weeks late, we have paid 40% advance,\n         want firm date without losing the relationship\nprompt:  <the actual text you would send a model>\ngood:    polite, gives a deadline, does not grovel,\n         natural Hindi not translated-English Hindi\nlast run: 2026-03-14  toolA 2  toolB 1  local-qwen 2"},{"t":"p","runs":[{"text":"The "},{"text":"good:","code":true},{"text":" line matters more than it looks. Writing down what a good answer contains "},{"text":"before","b":true},{"text":" you see any answers is what stops you rationalising afterwards. It takes two minutes per prompt and it is the difference between a test and a vibe."}]},{"t":"h2","id":"keeping-it-honest-over-time","runs":[{"text":"Keeping it honest over time"}]},{"t":"p","runs":[{"text":"Do not fix a prompt because a tool failed it.","b":true},{"text":" That is how a test set stops testing. If the prompt is badly written, rewrite it and reset the history, noting that you did."}]},{"t":"p","runs":[{"text":"Do not let it become easy.","b":true},{"text":" As you get better at prompting, your prompts get clearer, and clearer prompts are easier. Every six months, add one hard new item and retire nothing."}]},{"t":"p","runs":[{"text":"Keep the failures.","b":true},{"text":" When a tool gets something wrong in an interesting way, that becomes prompt thirteen."}]},{"t":"p","runs":[{"text":"Redact once, properly.","b":true},{"text":" These prompts will be pasted into other people’s services. Replace real names, account numbers and addresses with consistent stand-ins the first time you build the file, and you never have to think about it again. “The supplier” works exactly as well as the company’s name, and the answer is the same."}]},{"t":"h2","id":"what-it-costs-and-what-it-saves","runs":[{"text":"What it costs and what it saves"}]},{"t":"p","runs":[{"text":"Building it: one evening. Running it across three tools: about forty minutes, twice a year."}]},{"t":"p","runs":[{"text":"Against that, the thing it prevents. People switch tools on the strength of a launch video, spend a fortnight rebuilding habits, and discover the new one is worse at the one thing they do most. Or they stay on a paid plan for two years after a free tier overtook it. Forty minutes twice a year is a very cheap insurance policy against both."}]},{"t":"p","runs":[{"text":"There is one more use, and it is the one people are most grateful for later. When you doubt yourself ("},{"text":"has this got worse, or have I got fussier?","i":true},{"text":"), the file answers. Memory of software quality is unreliable in a specific direction: we remember the early impressive answers and forget the early failures, so everything feels like it has declined. The file is the only witness that is not your own recollection."}]}],"check":{"question":[{"text":"You write the twelve prompts and record what each answer should contain, but only after reading the first tool’s replies. What has that cost you?"}],"options":[{"text":[{"text":"Nothing, as long as the same criteria are then applied identically to every other tool."}],"correct":false,"tempting":[{"text":"Assumes fairness is only about consistency; criteria derived from one tool’s output describe that tool, so applying them evenly still measures how closely rivals imitate it."}]},{"text":[{"text":"The criteria now describe what the first tool produced, so the test measures similarity to it."}],"correct":true,"tempting":null},{"text":[{"text":"Only the ordering is affected, and rotating which tool goes first on the next run corrects it."}],"correct":false,"tempting":[{"text":"Treats an anchoring problem as an order effect; rotation helps with fatigue and ordering, but the criteria themselves stay contaminated."}]},{"text":[{"text":"The prompts are still valid but the scoring should be redone by a second person to remove your bias from the judgement."}],"correct":false,"tempting":[{"text":"Adds a reviewer to a standard that is already shaped by the first tool’s output, so a second person applies the same contaminated criteria."}]}],"order":[2,3,0,1],"why":null},"prev":{"slug":"real-differences","title":"Where they differ, and where it is marketing","minutes":8},"next":{"slug":"known-answer-probes","title":"The prompts where you already know the answer","minutes":8}}}