Nobody talks to a model
You talk to a product that talks to a model. Between your typing and the weights there is a stack of things you did not write, and it differs between ChatGPT, Claude, Gemini, Copilot, Perplexity, a WhatsApp bot your bank built, and a raw API call. When a prompt that worked for a colleague fails for you, the difference is usually here rather than in the words.
Five layers are worth knowing by name.
The system prompt. Every product prepends instructions about tone, refusals, formatting, and what tools exist. These run to thousands of words in consumer apps, and they are why the same underlying model sounds chatty in one product and clipped in another. You did not write them, you usually cannot see them, and they are in front of everything you say.
Tools. Web search, code execution, file reading, image generation, calendar access. A model with search can tell you today’s news; the identical model without it will either say it cannot or, worse, produce something plausible from training data. If you have ever seen a model get a live share price right in one app and invent one in another, this is why. When accuracy about anything recent matters, check whether the thing you are using can actually look.
Retrieval and attachments. Uploading a PDF does not put the PDF in front of the model. The product extracts text (sometimes running OCR on scans, sometimes silently failing on a scanned page) and may index it and inject only the passages it judges relevant to your question. Two products given the same file can therefore see different halves of it.
Memory. Several products now save facts about you across conversations and inject them. Useful when it remembers your job. Awkward when a stray sentence from March quietly shapes an answer in September and you have no idea it is in the prompt. Every product with memory has a page listing what it stored; read yours once. It is often surprising.
Routing. Some products decide per message which model handles it: a small fast one for easy questions, a large one for hard ones. So “the same model” may not even be the same model between two of your own messages, which makes informal comparisons unreliable.
What this changes in practice
Test in the place you will work. A prompt validated in a developer playground and deployed into a chat product is not validated. The system prompt alone can change the output shape.
Say the date and the locale if either matters. Some products inject today’s date; some do not. “As of today” means nothing if the model has no idea what today is. Write the date.
When a shared prompt fails, check the layers before rewriting the words. Does your version have search? Is the file being read fully? Is memory injecting something? Are you on the same model tier? Four questions, and one of them is usually the answer.
Expect refusal boundaries to differ. The same request can be answered in one product and declined in another, because the safety instructions live in the system prompt and in provider-specific training, not in the model alone.
A free way to see underneath
If you want to know what a prompt does without a product in the way, use an interface that shows you the whole conversation. Provider playgrounds do this, and they are usually free to open even if calls cost a small amount. For a fully free path, run a small open model on your own machine (ollama run llama3.2 on a laptop, or llama.cpp), where there is no system prompt except the one you write and no hidden tools. The answers will be weaker than a frontier model’s. That is not the point. The point is that you can see the entire input for the first time, and it makes everything in this module concrete.
The habit
When something surprising happens, ask two questions in order.
- What was in the block? System prompt, memory, retrieved passages, tool results, my message.
- What did the sampler do? Would a second run look different?
Almost every “the AI is being weird today” has an answer in one of those two places. Neither of them is the words you typed, which is precisely why people spend so long rewriting the words.
Every lesson here is open. Your progress is recorded once you pass the AI, Actually Explained exam.
Check your understanding
A team shares a prompt for checking supplier invoices against a price list. It works for the person who wrote it and returns “price not found” for everyone else. What should they check first?
Pick the one you would defend. Your answer stays private.
Stuck on something?Ask Asha, your AI tutor