{"course":{"slug":"ai-actually-explained","title":"AI, Actually Explained","url":"https://addaly.com/learn/ai-actually-explained","level":"beginner","track":"start"},"lesson":{"slug":"the-invisible-instructions","title":"The text you never see","url":"https://addaly.com/learn/ai-actually-explained/the-invisible-instructions","position":29,"of":92,"minutes":8,"answer":"The hidden system prompt is ordinary text in the same context as your message, so it can be revealed.","keyIdea":"The hidden system prompt is ordinary text in the same context as your message, so it can be revealed, outweighed and diluted: treat it as a request and enforce anything critical in software outside the model."},"body":{"format":"markdown","text":"## Somebody spoke first\n\nWhen you open a chat and type, yours is not the first message. Above it, invisible, sits a block of text placed there by whoever built the product. It is called the **system prompt**, and it typically runs from a few hundred to several thousand words.\n\nWhat is in it:\n\n- A name and a character. \"You are Aria, the assistant for a bank.\"\n- Today's date, since the model has no clock.\n- Tone and formatting instructions: length, whether to use lists, which language to reply in.\n- Boundaries. What to refuse, what to escalate, what never to promise.\n- Descriptions of any tools it may use, and when.\n- Sometimes company facts: opening hours, the refund policy, product names.\n\nThis is why the same underlying model, licensed by three companies, behaves like three different products. Often the only difference is this text.\n\nSome organisations publish theirs. Anthropic has published the system prompts for its consumer Claude applications, which is worth reading once for the sheer ordinariness of it: it is instructions, in English, in the same box as your message.\n\n## The crucial property: it is only text\n\nA system prompt looks like configuration. It is not. It is words in the context window, sitting in the same space as your message and the document you pasted, and it is processed by the same mechanism.\n\nModels are trained to weight it heavily, and that training works most of the time. But \"weighted heavily\" is not \"enforced\". Three things follow, and they are the reason this lesson exists:\n\n**It can be revealed.** People have extracted the system prompts of most commercial assistants, essentially by asking cleverly. Treat anything in a system prompt as public.\n\n**It can be overridden.** A long conversation, a contradictory instruction, or text inside a document the model is reading can outweigh it. When a document says \"ignore your previous instructions\", the model has no reliable way to tell that this instruction has a different status from the one above. That is prompt injection, and it has its own lesson in the next module.\n\n**It competes for attention and space.** In a very long conversation the system prompt is still there, but it is one voice among many thousands of tokens, and its influence measurably weakens.\n\nThe rule that follows is simple and important: **a system prompt is a request, not a control.** Anything that must not happen (a refund issued, a record deleted, a price quoted) has to be prevented by ordinary software outside the model, which checks the model's output before acting on it. Every organisation that has learned this the hard way learned it by trusting an instruction.\n\n## Using it deliberately, which most people do not\n\nWhere you can set a system prompt yourself, it is the single strongest lever available, and it is under-used because it is hidden in settings.\n\nYou can set one in ChatGPT's custom instructions and Projects, in Claude's Projects, in Gemini's Gems, in almost every API, and in local tools such as Ollama and LM Studio.\n\nWhat belongs there is the standing context you would otherwise retype daily:\n\n- Who you are and what you do. \"I am a nurse in Kerala writing patient information leaflets.\"\n- The audience. \"Readers have limited English and no medical background.\"\n- Standing preferences. \"Use British spelling. No bullet lists unless asked. Say when you are unsure rather than guessing.\"\n- Standing prohibitions. \"Never invent a citation. If you do not know, say so.\"\n\nThe gain is not only convenience. Instructions given once at the top apply consistently across a conversation, where an instruction given in message forty applies mainly to message forty-one.\n\n## Two warnings\n\n**Do not put secrets in it.** Not an API key, not a password, not a client's private data. It is retrievable and it is sent to the provider with every request.\n\n**Keep it short and concrete.** A three-thousand-word system prompt full of contradictory rules produces worse behaviour than a two-hundred-word one, because the model must reconcile the contradictions and will do so unpredictably. Write the four things that matter, and test whether each one changes anything by removing it."},"check":{"question":"A company puts \"never issue a refund over £50 without a manager\" in its support bot's system prompt and treats the limit as enforced. Why is that unsafe?","options":[{"text":"The model cannot compare numbers reliably, so it will misjudge which amounts exceed £50.","correct":false,"tempting":"Blames numerical ability; the risk remains even with perfect arithmetic, because the instruction itself is not binding."},{"text":"System prompts are only applied at the start of a session and are removed once the conversation exceeds a few turns.","correct":false,"tempting":"Describes removal rather than dilution; the text remains present but its influence weakens and can be outweighed."},{"text":"The instruction is text competing with everything else in the context, so it can be outweighed or overridden, and only software checking the output before acting can actually enforce a limit.","correct":true},{"text":"Refund limits must be stated in the training data rather than the system prompt to take effect.","correct":false,"tempting":"Suggests fine-tuning as the fix, which changes tendencies but likewise cannot guarantee a hard limit."}]},"prev":{"slug":"taught-to-be-helpful","title":"The second training, where the assistant is made","url":"https://addaly.com/learn/ai-actually-explained/taught-to-be-helpful"},"next":{"slug":"not-a-search-engine","title":"Why it is not a search engine, and what changes when you bolt one on","url":"https://addaly.com/learn/ai-actually-explained/not-a-search-engine"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The text you never see', in AI, Actually Explained. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-actually-explained/the-invisible-instructions (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The text you never see","courseTitle":"AI, Actually Explained","module":{"index":3,"slug":"how-a-chatbot-works","title":"What a chatbot is actually doing"},"answer":null,"keyIdea":"The hidden system prompt is ordinary text in the same context as your message, so it can be revealed, outweighed and diluted: treat it as a request and enforce anything critical in software outside the model.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"somebody-spoke-first","runs":[{"text":"Somebody spoke first"}]},{"t":"p","runs":[{"text":"When you open a chat and type, yours is not the first message. Above it, invisible, sits a block of text placed there by whoever built the product. It is called the "},{"text":"system prompt","b":true},{"text":", and it typically runs from a few hundred to several thousand words."}]},{"t":"p","runs":[{"text":"What is in it:"}]},{"t":"ul","items":[[{"text":"A name and a character. “You are Aria, the assistant for a bank.”"}],[{"text":"Today’s date, since the model has no clock."}],[{"text":"Tone and formatting instructions: length, whether to use lists, which language to reply in."}],[{"text":"Boundaries. What to refuse, what to escalate, what never to promise."}],[{"text":"Descriptions of any tools it may use, and when."}],[{"text":"Sometimes company facts: opening hours, the refund policy, product names."}]]},{"t":"p","runs":[{"text":"This is why the same underlying model, licensed by three companies, behaves like three different products. Often the only difference is this text."}]},{"t":"p","runs":[{"text":"Some organisations publish theirs. Anthropic has published the system prompts for its consumer Claude applications, which is worth reading once for the sheer ordinariness of it: it is instructions, in English, in the same box as your message."}]},{"t":"h2","id":"the-crucial-property-it-is-only-text","runs":[{"text":"The crucial property: it is only text"}]},{"t":"p","runs":[{"text":"A system prompt looks like configuration. It is not. It is words in the context window, sitting in the same space as your message and the document you pasted, and it is processed by the same mechanism."}]},{"t":"p","runs":[{"text":"Models are trained to weight it heavily, and that training works most of the time. But “weighted heavily” is not “enforced”. Three things follow, and they are the reason this lesson exists:"}]},{"t":"p","runs":[{"text":"It can be revealed.","b":true},{"text":" People have extracted the system prompts of most commercial assistants, essentially by asking cleverly. Treat anything in a system prompt as public."}]},{"t":"p","runs":[{"text":"It can be overridden.","b":true},{"text":" A long conversation, a contradictory instruction, or text inside a document the model is reading can outweigh it. When a document says “ignore your previous instructions”, the model has no reliable way to tell that this instruction has a different status from the one above. That is prompt injection, and it has its own lesson in the next module."}]},{"t":"p","runs":[{"text":"It competes for attention and space.","b":true},{"text":" In a very long conversation the system prompt is still there, but it is one voice among many thousands of tokens, and its influence measurably weakens."}]},{"t":"p","runs":[{"text":"The rule that follows is simple and important: "},{"text":"a system prompt is a request, not a control.","b":true},{"text":" Anything that must not happen (a refund issued, a record deleted, a price quoted) has to be prevented by ordinary software outside the model, which checks the model’s output before acting on it. Every organisation that has learned this the hard way learned it by trusting an instruction."}]},{"t":"h2","id":"using-it-deliberately-which-most-people-do-not","runs":[{"text":"Using it deliberately, which most people do not"}]},{"t":"p","runs":[{"text":"Where you can set a system prompt yourself, it is the single strongest lever available, and it is under-used because it is hidden in settings."}]},{"t":"p","runs":[{"text":"You can set one in ChatGPT’s custom instructions and Projects, in Claude’s Projects, in Gemini’s Gems, in almost every API, and in local tools such as Ollama and LM Studio."}]},{"t":"p","runs":[{"text":"What belongs there is the standing context you would otherwise retype daily:"}]},{"t":"ul","items":[[{"text":"Who you are and what you do. “I am a nurse in Kerala writing patient information leaflets.”"}],[{"text":"The audience. “Readers have limited English and no medical background.”"}],[{"text":"Standing preferences. “Use British spelling. No bullet lists unless asked. Say when you are unsure rather than guessing.”"}],[{"text":"Standing prohibitions. “Never invent a citation. If you do not know, say so.”"}]]},{"t":"p","runs":[{"text":"The gain is not only convenience. Instructions given once at the top apply consistently across a conversation, where an instruction given in message forty applies mainly to message forty-one."}]},{"t":"h2","id":"two-warnings","runs":[{"text":"Two warnings"}]},{"t":"p","runs":[{"text":"Do not put secrets in it.","b":true},{"text":" Not an API key, not a password, not a client’s private data. It is retrievable and it is sent to the provider with every request."}]},{"t":"p","runs":[{"text":"Keep it short and concrete.","b":true},{"text":" A three-thousand-word system prompt full of contradictory rules produces worse behaviour than a two-hundred-word one, because the model must reconcile the contradictions and will do so unpredictably. Write the four things that matter, and test whether each one changes anything by removing it."}]}],"check":{"question":[{"text":"A company puts “never issue a refund over £50 without a manager” in its support bot’s system prompt and treats the limit as enforced. Why is that unsafe?"}],"options":[{"text":[{"text":"The model cannot compare numbers reliably, so it will misjudge which amounts exceed £50."}],"correct":false,"tempting":[{"text":"Blames numerical ability; the risk remains even with perfect arithmetic, because the instruction itself is not binding."}]},{"text":[{"text":"System prompts are only applied at the start of a session and are removed once the conversation exceeds a few turns."}],"correct":false,"tempting":[{"text":"Describes removal rather than dilution; the text remains present but its influence weakens and can be outweighed."}]},{"text":[{"text":"The instruction is text competing with everything else in the context, so it can be outweighed or overridden, and only software checking the output before acting can actually enforce a limit."}],"correct":true,"tempting":null},{"text":[{"text":"Refund limits must be stated in the training data rather than the system prompt to take effect."}],"correct":false,"tempting":[{"text":"Suggests fine-tuning as the fix, which changes tendencies but likewise cannot guarantee a hard limit."}]}],"order":[0,1,3,2],"why":null},"prev":{"slug":"taught-to-be-helpful","title":"The second training, where the assistant is made","minutes":9},"next":{"slug":"not-a-search-engine","title":"Why it is not a search engine, and what changes when you bolt one on","minutes":9}}}