{"course":{"slug":"prompting","title":"Getting Real Work out of a Model","url":"https://addaly.com/learn/prompting","level":"beginner","track":"start"},"lesson":{"slug":"the-product-around-the-model","title":"Why the same prompt behaves differently in different apps","url":"https://addaly.com/learn/prompting/the-product-around-the-model","position":9,"of":72,"minutes":8,"answer":"Your prompt is only one layer of what reaches the model, so diagnose surprising behaviour by asking what else was in the block before you rewrite what you wrote.","keyIdea":"Your prompt is only one layer of what reaches the model, so diagnose surprising behaviour by asking what else was in the block before you rewrite what you wrote."},"body":{"format":"markdown","text":"## Nobody talks to a model\n\nYou talk to a product that talks to a model. Between your typing and the weights there is a stack of things you did not write, and it differs between ChatGPT, Claude, Gemini, Copilot, Perplexity, a WhatsApp bot your bank built, and a raw API call. When a prompt that worked for a colleague fails for you, the difference is usually here rather than in the words.\n\nFive layers are worth knowing by name.\n\n**The system prompt.** Every product prepends instructions about tone, refusals, formatting, and what tools exist. These run to thousands of words in consumer apps, and they are why the same underlying model sounds chatty in one product and clipped in another. You did not write them, you usually cannot see them, and they are in front of everything you say.\n\n**Tools.** Web search, code execution, file reading, image generation, calendar access. A model with search can tell you today's news; the identical model without it will either say it cannot or, worse, produce something plausible from training data. If you have ever seen a model get a live share price right in one app and invent one in another, this is why. When accuracy about anything recent matters, check whether the thing you are using can actually look.\n\n**Retrieval and attachments.** Uploading a PDF does not put the PDF in front of the model. The product extracts text (sometimes running OCR on scans, sometimes silently failing on a scanned page) and may index it and inject only the passages it judges relevant to your question. Two products given the same file can therefore see different halves of it.\n\n**Memory.** Several products now save facts about you across conversations and inject them. Useful when it remembers your job. Awkward when a stray sentence from March quietly shapes an answer in September and you have no idea it is in the prompt. Every product with memory has a page listing what it stored; read yours once. It is often surprising.\n\n**Routing.** Some products decide per message which model handles it: a small fast one for easy questions, a large one for hard ones. So \"the same model\" may not even be the same model between two of your own messages, which makes informal comparisons unreliable.\n\n> Figure: What sits between your typing and the model. Layers, top to bottom: System prompt (thousands of words on tone, refusals and tools, in front of everything you say); Tools (search, code, file reading; present in one app and absent in another); Retrieval and attachments (extracted text, or only the chunks judged relevant); Memory (notes saved about you, injected when they seem relevant); Routing (which model actually answers this message); Your message (the only layer you can see). Five layers you did not write. When a colleague's prompt fails for you, the difference is usually in one of these rather than in the words.\n\n## What this changes in practice\n\n**Test in the place you will work.** A prompt validated in a developer playground and deployed into a chat product is not validated. The system prompt alone can change the output shape.\n\n**Say the date and the locale if either matters.** Some products inject today's date; some do not. \"As of today\" means nothing if the model has no idea what today is. Write the date.\n\n**When a shared prompt fails, check the layers before rewriting the words.** Does your version have search? Is the file being read fully? Is memory injecting something? Are you on the same model tier? Four questions, and one of them is usually the answer.\n\n**Expect refusal boundaries to differ.** The same request can be answered in one product and declined in another, because the safety instructions live in the system prompt and in provider-specific training, not in the model alone.\n\n## A free way to see underneath\n\nIf you want to know what a prompt does without a product in the way, use an interface that shows you the whole conversation. Provider playgrounds do this, and they are usually free to open even if calls cost a small amount. For a fully free path, run a small open model on your own machine (`ollama run llama3.2` on a laptop, or `llama.cpp`), where there is no system prompt except the one you write and no hidden tools. The answers will be weaker than a frontier model's. That is not the point. The point is that you can see the entire input for the first time, and it makes everything in this module concrete.\n\n## The habit\n\nWhen something surprising happens, ask two questions in order.\n\n1. **What was in the block?** System prompt, memory, retrieved passages, tool results, my message.\n2. **What did the sampler do?** Would a second run look different?\n\nAlmost every \"the AI is being weird today\" has an answer in one of those two places. Neither of them is the words you typed, which is precisely why people spend so long rewriting the words."},"check":{"question":"A team shares a prompt for checking supplier invoices against a price list. It works for the person who wrote it and returns \"price not found\" for everyone else. What should they check first?","options":[{"text":"Whether the price list is actually reaching the others: the author may have it in an attached file, a memory entry, or a tool the others do not have enabled.","correct":true},{"text":"Whether the others are typing the prompt slightly differently, since small wording changes matter.","correct":false,"tempting":"Possible but unlikely to produce a clean split between one working user and all the others; a uniform failure points at a uniform difference in setup."},{"text":"Whether the model version changed between when it was written and when it was shared.","correct":false,"tempting":"A model update would tend to affect the author as well, not everyone except the author."},{"text":"Whether the others are hitting rate limits that truncate the response.","correct":false,"tempting":"Rate limiting produces errors or cut-off text, not a coherent \"not found\" answer."}]},"prev":{"slug":"specificity-and-its-limit","title":"How much detail is too much","url":"https://addaly.com/learn/prompting/specificity-and-its-limit"},"next":{"slug":"context-is-most-of-the-job","title":"Context is most of the job","url":"https://addaly.com/learn/prompting/context-is-most-of-the-job"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Why the same prompt behaves differently in different apps', in Getting Real Work out of a Model. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/prompting/the-product-around-the-model (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Why the same prompt behaves differently in different apps","courseTitle":"Getting Real Work out of a Model","module":{"index":1,"slug":"what-a-prompt-is","title":"What a prompt actually is"},"answer":null,"keyIdea":"Your prompt is only one layer of what reaches the model, so diagnose surprising behaviour by asking what else was in the block before you rewrite what you wrote.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"stack","title":"What sits between your typing and the model","layers":[{"label":"System prompt","detail":"thousands of words on tone, refusals and tools, in front of everything you say"},{"label":"Tools","detail":"search, code, file reading; present in one app and absent in another"},{"label":"Retrieval and attachments","detail":"extracted text, or only the chunks judged relevant"},{"label":"Memory","detail":"notes saved about you, injected when they seem relevant"},{"label":"Routing","detail":"which model actually answers this message"},{"label":"Your message","detail":"the only layer you can see"}],"caption":"Five layers you did not write. When a colleague’s prompt fails for you, the difference is usually in one of these rather than in the words."},"alt":"Figure: What sits between your typing and the model. Layers, top to bottom: System prompt (thousands of words on tone, refusals and tools, in front of everything you say); Tools (search, code, file reading; present in one app and absent in another); Retrieval and attachments (extracted text, or only the chunks judged relevant); Memory (notes saved about you, injected when they seem relevant); Routing (which model actually answers this message); Your message (the only layer you can see). Five layers you did not write. When a colleague’s prompt fails for you, the difference is usually in one of these rather than in the words."}},"blocks":[{"t":"h2","id":"nobody-talks-to-a-model","runs":[{"text":"Nobody talks to a model"}]},{"t":"p","runs":[{"text":"You talk to a product that talks to a model. Between your typing and the weights there is a stack of things you did not write, and it differs between ChatGPT, Claude, Gemini, Copilot, Perplexity, a WhatsApp bot your bank built, and a raw API call. When a prompt that worked for a colleague fails for you, the difference is usually here rather than in the words."}]},{"t":"p","runs":[{"text":"Five layers are worth knowing by name."}]},{"t":"p","runs":[{"text":"The system prompt.","b":true},{"text":" Every product prepends instructions about tone, refusals, formatting, and what tools exist. These run to thousands of words in consumer apps, and they are why the same underlying model sounds chatty in one product and clipped in another. You did not write them, you usually cannot see them, and they are in front of everything you say."}]},{"t":"p","runs":[{"text":"Tools.","b":true},{"text":" Web search, code execution, file reading, image generation, calendar access. A model with search can tell you today’s news; the identical model without it will either say it cannot or, worse, produce something plausible from training data. If you have ever seen a model get a live share price right in one app and invent one in another, this is why. When accuracy about anything recent matters, check whether the thing you are using can actually look."}]},{"t":"p","runs":[{"text":"Retrieval and attachments.","b":true},{"text":" Uploading a PDF does not put the PDF in front of the model. The product extracts text (sometimes running OCR on scans, sometimes silently failing on a scanned page) and may index it and inject only the passages it judges relevant to your question. Two products given the same file can therefore see different halves of it."}]},{"t":"p","runs":[{"text":"Memory.","b":true},{"text":" Several products now save facts about you across conversations and inject them. Useful when it remembers your job. Awkward when a stray sentence from March quietly shapes an answer in September and you have no idea it is in the prompt. Every product with memory has a page listing what it stored; read yours once. It is often surprising."}]},{"t":"p","runs":[{"text":"Routing.","b":true},{"text":" Some products decide per message which model handles it: a small fast one for easy questions, a large one for hard ones. So “the same model” may not even be the same model between two of your own messages, which makes informal comparisons unreliable."}]},{"t":"h2","id":"what-this-changes-in-practice","runs":[{"text":"What this changes in practice"}]},{"t":"p","runs":[{"text":"Test in the place you will work.","b":true},{"text":" A prompt validated in a developer playground and deployed into a chat product is not validated. The system prompt alone can change the output shape."}]},{"t":"p","runs":[{"text":"Say the date and the locale if either matters.","b":true},{"text":" Some products inject today’s date; some do not. “As of today” means nothing if the model has no idea what today is. Write the date."}]},{"t":"p","runs":[{"text":"When a shared prompt fails, check the layers before rewriting the words.","b":true},{"text":" Does your version have search? Is the file being read fully? Is memory injecting something? Are you on the same model tier? Four questions, and one of them is usually the answer."}]},{"t":"p","runs":[{"text":"Expect refusal boundaries to differ.","b":true},{"text":" The same request can be answered in one product and declined in another, because the safety instructions live in the system prompt and in provider-specific training, not in the model alone."}]},{"t":"h2","id":"a-free-way-to-see-underneath","runs":[{"text":"A free way to see underneath"}]},{"t":"p","runs":[{"text":"If you want to know what a prompt does without a product in the way, use an interface that shows you the whole conversation. Provider playgrounds do this, and they are usually free to open even if calls cost a small amount. For a fully free path, run a small open model on your own machine ("},{"text":"ollama run llama3.2","code":true},{"text":" on a laptop, or "},{"text":"llama.cpp","code":true},{"text":"), where there is no system prompt except the one you write and no hidden tools. The answers will be weaker than a frontier model’s. That is not the point. The point is that you can see the entire input for the first time, and it makes everything in this module concrete."}]},{"t":"h2","id":"the-habit","runs":[{"text":"The habit"}]},{"t":"p","runs":[{"text":"When something surprising happens, ask two questions in order."}]},{"t":"ol","start":1,"items":[[{"text":"What was in the block?","b":true},{"text":" System prompt, memory, retrieved passages, tool results, my message."}],[{"text":"What did the sampler do?","b":true},{"text":" Would a second run look different?"}]]},{"t":"p","runs":[{"text":"Almost every “the AI is being weird today” has an answer in one of those two places. Neither of them is the words you typed, which is precisely why people spend so long rewriting the words."}]}],"check":{"question":[{"text":"A team shares a prompt for checking supplier invoices against a price list. It works for the person who wrote it and returns “price not found” for everyone else. What should they check first?"}],"options":[{"text":[{"text":"Whether the price list is actually reaching the others: the author may have it in an attached file, a memory entry, or a tool the others do not have enabled."}],"correct":true,"tempting":null},{"text":[{"text":"Whether the others are typing the prompt slightly differently, since small wording changes matter."}],"correct":false,"tempting":[{"text":"Possible but unlikely to produce a clean split between one working user and all the others; a uniform failure points at a uniform difference in setup."}]},{"text":[{"text":"Whether the model version changed between when it was written and when it was shared."}],"correct":false,"tempting":[{"text":"A model update would tend to affect the author as well, not everyone except the author."}]},{"text":[{"text":"Whether the others are hitting rate limits that truncate the response."}],"correct":false,"tempting":[{"text":"Rate limiting produces errors or cut-off text, not a coherent “not found” answer."}]}],"order":[3,1,2,0],"why":null},"prev":{"slug":"specificity-and-its-limit","title":"How much detail is too much","minutes":8},"next":{"slug":"context-is-most-of-the-job","title":"Context is most of the job","minutes":7}}}