System prompts, user turns, and who the model listens to

Three roles, one blob of text

A request carries a system prompt and a list of turns with roles: user and assistant. Underneath, the model sees one long sequence. The roles are structure and emphasis, not walls.

The system prompt is where you put things that are true for every request: what this feature is, what format you want, what to do when the input is unclear, what never to do.

r = client.messages.create(
    model=MODEL,
    max_tokens=16,
    system=(
        "You classify customer messages for a bus ticketing company in Lagos.\n"
        "Reply with exactly one word: REFUND, SCHEDULE, COMPLAINT, or OTHER.\n"
        "If the message is ambiguous or off-topic, reply OTHER.\n"
        "Never explain your answer."
    ),
    messages=[
        {"role": "user", "content": "My bus to Ibadan never came and nobody answered the hotline."},
        {"role": "assistant", "content": "COMPLAINT"},
        {"role": "user", "content": "I paid twice by mistake, can I get the second one back?"},
        {"role": "assistant", "content": "REFUND"},
        {"role": "user", "content": incoming_message},
    ],
)

Look at that carefully. The first four messages never happened. You wrote both sides. This is few-shot prompting, and fabricating turns is the normal way to do it — the model treats them as evidence of how this conversation goes. Two or three examples, chosen to cover the cases you get wrong, beat two paragraphs of instructions describing the same thing.

Write rules as behaviour, not adjectives

"Be concise and professional" gives the model almost nothing. "Reply in under 40 words. No greeting. No apology unless the customer reports a loss." gives it a target it can hit and you can test.

The same applies to edge cases. Every classifier needs an escape hatch, or it will invent a category rather than fail. "If unclear, reply OTHER" is doing real work in the example above.

Where you put the long document matters

When you are passing a long document, put the document first and the question last. Models attend more reliably to instructions near the end of a long input, and burying your question above 20 pages of text is a common cause of "it ignored what I asked".

user_turn = f"{contract_text}\n\n---\n\nList every payment deadline in the contract above, with its clause number."

The system prompt is not a security boundary

This is the part people learn the expensive way. A system prompt is the strongest thing you say. It is not a rule the model cannot break.

The user's text lands in the same context window as your instructions. A determined user, or a document your feature was asked to summarise, can argue with your rules, and sometimes wins. Refusals are trained behaviour, not enforcement.

So split your rules into two piles:

• Steering. Tone, format, what to do with ambiguity, which topics to stay on. The system prompt is the right place. Occasional slips are survivable.
• Enforcement. Who may see this record. Whether this refund is issued. What the maximum discount is. This belongs in your code, checked against the session, and it must hold even if the model does something strange.

A useful test: for each line in your system prompt, ask what happens if the model ignores it exactly once. If the answer is "a slightly worse reply", the prompt is fine. If the answer is "we leaked someone's phone number", it needs to be code.

Version it like code

Your prompt is program logic written in English. Put it in a file, not an f-string buried in a handler. Give it a version number. When behaviour changes and nobody deployed code, someone edited a prompt, and you want to be able to see the diff.