{"course":{"slug":"prompting","title":"Getting Real Work out of a Model","url":"https://addaly.com/learn/prompting","level":"beginner","track":"start"},"lesson":{"slug":"diagnosing-a-bad-answer","title":"Four reasons an answer is bad","url":"https://addaly.com/learn/prompting/diagnosing-a-bad-answer","position":47,"of":72,"minutes":9,"answer":"Bad answers have four causes (missing material, misread instruction, wrong output form, and a task beyond the tool) and the giveaway for the fourth is that better prompting produces a better-looking wrong answer.","keyIdea":"Bad answers have four causes (missing material, misread instruction, wrong output form, and a task beyond the tool) and the giveaway for the fourth is that better prompting produces a better-looking wrong answer."},"body":{"format":"markdown","text":"## Name the failure before you fix it\n\nThe instinct on receiving a poor answer is to rewrite the prompt and try again. That works often enough to be a habit and slowly enough to be a waste, because most of the rewriting addresses the wrong thing.\n\nThere are four causes. They need four different responses, and telling them apart takes about ten seconds once you know what to look for.\n\n## 1. It did not have what it needed\n\n**Symptom:** the answer is generic, hedged, or true of a category rather than your case. Phrases like \"depending on your situation\" and \"you may want to consider\". It answers a question adjacent to yours.\n\n**Test:** could a competent stranger have produced a better answer from this text alone? If not, the material was missing.\n\n**Fix:** paste the thing. The document, the error, the data, the constraint that makes it hard. Do not rewrite the instruction; it was probably fine.\n\nThis is by a wide margin the most common cause, and the one people least often diagnose, because a generic answer reads like a lack of effort rather than a lack of information.\n\n## 2. It did not understand what you wanted\n\n**Symptom:** a good answer to a different question. A summary when you wanted an assessment. Advice when you wanted options. It went one level up or down from where you were aiming.\n\n**Test:** read your instruction as a stranger. Is there a second reasonable reading? There usually is, and the model took it.\n\n**Fix:** name the verb, the object and the finished state. \"Review this\" becomes \"find every clause that lets them cancel without paying, quote each one, and say in one sentence what it means for me.\"\n\n## 3. The output is in an unusable form\n\n**Symptom:** the content is right and you have work to do before you can use it. Nine paragraphs where you wanted a table. A table where you wanted a message you could send. Padding around the useful part.\n\n**Test:** would you accept this if it were laid out differently?\n\n**Fix:** the format lesson. Say the container, say what to omit, say what to do about gaps. Nothing about the substance needs to change.\n\n## 4. It cannot do this\n\n**Symptom:** the answer is confidently wrong in a way that does not improve when you supply more context or clearer instructions. Arithmetic that stays wrong. Claims about recent events. Precise citations. Counting things inside words. Knowledge of your private systems.\n\n**Test:** you have made two genuine improvements and the same class of error survives both.\n\n**Fix:** change the approach, not the prompt. Give it a calculator, a search tool, the document itself. Or do that part yourself.\n\nThe distinguishing mark of cause four is that better prompting produces a better-*looking* wrong answer. If your third attempt is more polished and equally wrong, stop rewriting.\n\n> Figure: Where bad answers come from. Bars (% of bad answers, roughly): 1. Missing material 55; 2. Misread instruction 25; 3. Unusable form 15; 4. Beyond the tool 5. Two of the four are thirty-second fixes and account for most of what anybody gets. People work from the bottom up, concluding the tool is not good enough while their prompt still contains no material.\n\n## A worked diagnosis\n\n*\"I asked it to check my contract and it gave me a list of general things to look for in contracts.\"*\n\nCause one. The contract was described, not pasted.\n\n*\"I pasted the contract and it summarised it. I wanted the risks.\"*\n\nCause two. \"Check\" has several readings; it took the most common.\n\n*\"I asked for the risks and got six paragraphs. I wanted something I could take to my lawyer.\"*\n\nCause three. Content right, form wrong. Ask for a numbered list of questions with the clause quoted beside each.\n\n*\"It quoted a clause number that does not exist in my contract.\"*\n\nCause four, in its most dangerous form, and the fix is not a better prompt but a requirement that every claim carry a quote you can search for.\n\n## Why the order matters\n\nWork through them in order. Context first, because it is the commonest and the cheapest to fix. Then the instruction. Then the format. Only then consider that the task may be out of reach.\n\nPeople do it backwards: they conclude the model is not good enough while their prompt still contains no material. Two of the four causes are things you can fix in thirty seconds, and they account for most bad answers anybody gets.\n\n## The fifth thing, which is not a cause\n\nSometimes the answer is fine and you do not like it. It is accurate, well-formed, addresses what you asked, and says something you did not want to hear: the plan has a problem, the deadline is not achievable, the letter is weaker than you thought.\n\nThis is worth separating from the four causes, because it produces the same feeling and needs the opposite response. The tell is that you cannot say what is wrong with it, only that it is not what you were after. Pushing back here will get you agreement, because you already know what the training rewards, and the agreement will not be evidence of anything."},"check":{"question":"A user asks three times for a summary of a scientific paper and each time gets a fluent summary containing a finding the paper does not report. He has pasted the full paper and stated the task clearly. Which cause is this, and what follows?","options":[{"text":"Cause two: the instruction is still ambiguous and needs a sharper verb.","correct":false,"tempting":"Two clear attempts with the material present rule this out; a sharper verb would produce a differently phrased invention."},{"text":"Cause four in its quiet form: repeated prompting is producing better-looking wrong answers, so the fix is to require a quoted sentence for every claim rather than to rewrite the prompt again.","correct":true},{"text":"Cause one: the paper is present but the model needs more surrounding context about the field.","correct":false,"tempting":"Adding background does not address a claim that is contradicted by the pasted source."},{"text":"Cause three: the summary format allows too much freedom, so a stricter template will fix it.","correct":false,"tempting":"A template constrains shape, and an invented finding fits neatly inside any shape."}]},"prev":{"slug":"iterate-dont-reroll","title":"Edit the prompt, do not spin the wheel","url":"https://addaly.com/learn/prompting/iterate-dont-reroll"},"next":{"slug":"change-one-thing","title":"Change one thing at a time","url":"https://addaly.com/learn/prompting/change-one-thing"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Four reasons an answer is bad', in Getting Real Work out of a Model. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/prompting/diagnosing-a-bad-answer (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Four reasons an answer is bad","courseTitle":"Getting Real Work out of a Model","module":{"index":6,"slug":"from-lucky-to-reliable","title":"From lucky to reliable"},"answer":null,"keyIdea":"Bad answers have four causes (missing material, misread instruction, wrong output form, and a task beyond the tool) and the giveaway for the fourth is that better prompting produces a better-looking wrong answer.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"bars","unit":"% of bad answers, roughly","title":"Where bad answers come from","series":[{"label":"1. Missing material","value":55,"highlight":true},{"label":"2. Misread instruction","value":25},{"label":"3. Unusable form","value":15},{"label":"4. Beyond the tool","value":5}],"caption":"Two of the four are thirty-second fixes and account for most of what anybody gets. People work from the bottom up, concluding the tool is not good enough while their prompt still contains no material."},"alt":"Figure: Where bad answers come from. Bars (% of bad answers, roughly): 1. Missing material 55; 2. Misread instruction 25; 3. Unusable form 15; 4. Beyond the tool 5. Two of the four are thirty-second fixes and account for most of what anybody gets. People work from the bottom up, concluding the tool is not good enough while their prompt still contains no material."}},"blocks":[{"t":"h2","id":"name-the-failure-before-you-fix-it","runs":[{"text":"Name the failure before you fix it"}]},{"t":"p","runs":[{"text":"The instinct on receiving a poor answer is to rewrite the prompt and try again. That works often enough to be a habit and slowly enough to be a waste, because most of the rewriting addresses the wrong thing."}]},{"t":"p","runs":[{"text":"There are four causes. They need four different responses, and telling them apart takes about ten seconds once you know what to look for."}]},{"t":"h2","id":"1-it-did-not-have-what-it-needed","runs":[{"text":"1. It did not have what it needed"}]},{"t":"p","runs":[{"text":"Symptom:","b":true},{"text":" the answer is generic, hedged, or true of a category rather than your case. Phrases like “depending on your situation” and “you may want to consider”. It answers a question adjacent to yours."}]},{"t":"p","runs":[{"text":"Test:","b":true},{"text":" could a competent stranger have produced a better answer from this text alone? If not, the material was missing."}]},{"t":"p","runs":[{"text":"Fix:","b":true},{"text":" paste the thing. The document, the error, the data, the constraint that makes it hard. Do not rewrite the instruction; it was probably fine."}]},{"t":"p","runs":[{"text":"This is by a wide margin the most common cause, and the one people least often diagnose, because a generic answer reads like a lack of effort rather than a lack of information."}]},{"t":"h2","id":"2-it-did-not-understand-what-you-wanted","runs":[{"text":"2. It did not understand what you wanted"}]},{"t":"p","runs":[{"text":"Symptom:","b":true},{"text":" a good answer to a different question. A summary when you wanted an assessment. Advice when you wanted options. It went one level up or down from where you were aiming."}]},{"t":"p","runs":[{"text":"Test:","b":true},{"text":" read your instruction as a stranger. Is there a second reasonable reading? There usually is, and the model took it."}]},{"t":"p","runs":[{"text":"Fix:","b":true},{"text":" name the verb, the object and the finished state. “Review this” becomes “find every clause that lets them cancel without paying, quote each one, and say in one sentence what it means for me.”"}]},{"t":"h2","id":"3-the-output-is-in-an-unusable-form","runs":[{"text":"3. The output is in an unusable form"}]},{"t":"p","runs":[{"text":"Symptom:","b":true},{"text":" the content is right and you have work to do before you can use it. Nine paragraphs where you wanted a table. A table where you wanted a message you could send. Padding around the useful part."}]},{"t":"p","runs":[{"text":"Test:","b":true},{"text":" would you accept this if it were laid out differently?"}]},{"t":"p","runs":[{"text":"Fix:","b":true},{"text":" the format lesson. Say the container, say what to omit, say what to do about gaps. Nothing about the substance needs to change."}]},{"t":"h2","id":"4-it-cannot-do-this","runs":[{"text":"4. It cannot do this"}]},{"t":"p","runs":[{"text":"Symptom:","b":true},{"text":" the answer is confidently wrong in a way that does not improve when you supply more context or clearer instructions. Arithmetic that stays wrong. Claims about recent events. Precise citations. Counting things inside words. Knowledge of your private systems."}]},{"t":"p","runs":[{"text":"Test:","b":true},{"text":" you have made two genuine improvements and the same class of error survives both."}]},{"t":"p","runs":[{"text":"Fix:","b":true},{"text":" change the approach, not the prompt. Give it a calculator, a search tool, the document itself. Or do that part yourself."}]},{"t":"p","runs":[{"text":"The distinguishing mark of cause four is that better prompting produces a better-"},{"text":"looking","i":true},{"text":" wrong answer. If your third attempt is more polished and equally wrong, stop rewriting."}]},{"t":"h2","id":"a-worked-diagnosis","runs":[{"text":"A worked diagnosis"}]},{"t":"p","runs":[{"text":"”I asked it to check my contract and it gave me a list of general things to look for in contracts.”","i":true}]},{"t":"p","runs":[{"text":"Cause one. The contract was described, not pasted."}]},{"t":"p","runs":[{"text":"”I pasted the contract and it summarised it. I wanted the risks.”","i":true}]},{"t":"p","runs":[{"text":"Cause two. “Check” has several readings; it took the most common."}]},{"t":"p","runs":[{"text":"”I asked for the risks and got six paragraphs. I wanted something I could take to my lawyer.”","i":true}]},{"t":"p","runs":[{"text":"Cause three. Content right, form wrong. Ask for a numbered list of questions with the clause quoted beside each."}]},{"t":"p","runs":[{"text":"”It quoted a clause number that does not exist in my contract.”","i":true}]},{"t":"p","runs":[{"text":"Cause four, in its most dangerous form, and the fix is not a better prompt but a requirement that every claim carry a quote you can search for."}]},{"t":"h2","id":"why-the-order-matters","runs":[{"text":"Why the order matters"}]},{"t":"p","runs":[{"text":"Work through them in order. Context first, because it is the commonest and the cheapest to fix. Then the instruction. Then the format. Only then consider that the task may be out of reach."}]},{"t":"p","runs":[{"text":"People do it backwards: they conclude the model is not good enough while their prompt still contains no material. Two of the four causes are things you can fix in thirty seconds, and they account for most bad answers anybody gets."}]},{"t":"h2","id":"the-fifth-thing-which-is-not-a-cause","runs":[{"text":"The fifth thing, which is not a cause"}]},{"t":"p","runs":[{"text":"Sometimes the answer is fine and you do not like it. It is accurate, well-formed, addresses what you asked, and says something you did not want to hear: the plan has a problem, the deadline is not achievable, the letter is weaker than you thought."}]},{"t":"p","runs":[{"text":"This is worth separating from the four causes, because it produces the same feeling and needs the opposite response. The tell is that you cannot say what is wrong with it, only that it is not what you were after. Pushing back here will get you agreement, because you already know what the training rewards, and the agreement will not be evidence of anything."}]}],"check":{"question":[{"text":"A user asks three times for a summary of a scientific paper and each time gets a fluent summary containing a finding the paper does not report. He has pasted the full paper and stated the task clearly. Which cause is this, and what follows?"}],"options":[{"text":[{"text":"Cause two: the instruction is still ambiguous and needs a sharper verb."}],"correct":false,"tempting":[{"text":"Two clear attempts with the material present rule this out; a sharper verb would produce a differently phrased invention."}]},{"text":[{"text":"Cause four in its quiet form: repeated prompting is producing better-looking wrong answers, so the fix is to require a quoted sentence for every claim rather than to rewrite the prompt again."}],"correct":true,"tempting":null},{"text":[{"text":"Cause one: the paper is present but the model needs more surrounding context about the field."}],"correct":false,"tempting":[{"text":"Adding background does not address a claim that is contradicted by the pasted source."}]},{"text":[{"text":"Cause three: the summary format allows too much freedom, so a stricter template will fix it."}],"correct":false,"tempting":[{"text":"A template constrains shape, and an invented finding fits neatly inside any shape."}]}],"order":[2,0,1,3],"why":null},"prev":{"slug":"iterate-dont-reroll","title":"Edit the prompt, do not spin the wheel","minutes":6},"next":{"slug":"change-one-thing","title":"Change one thing at a time","minutes":8}}}