What changes when nothing is hidden

You are now the scaffolding

A commercial chat product is not just a model. It is a model plus a long system prompt you never see, plus server-side reformatting, plus a tool harness, plus a safety layer, plus retries. When you download weights, you get the model and none of the rest.

That is the trade this whole course is about, and in prompting it shows up as a set of specific differences.

No hidden system prompt. Whatever behaviour the product had (cite sources, ask clarifying questions, refuse this category, format with headings) was partly instruction, not weights. You have to write it.

No reformatting. The exact string you send is what the model sees, special tokens and all. There is no service quietly fixing your template.

Weaker system-prompt adherence at small sizes. A 4B model follows a system prompt for a while and then drifts back to its tuned defaults. This is not a bug you can prompt away; it is what a smaller instruction-tuned model does.

Six things that measurably help

Put the critical constraint last. Attention to recent tokens is stronger, and small models in particular follow the end of the prompt more reliably than the middle. If one rule must hold, restate it immediately before the model generates.

Write positively, not negatively. Do not mention the price frequently produces a mention of the price, because the phrase puts the concept in context and small models are weak at the logical operation of suppression. Discuss only availability and delivery works better. This is one of the most reliable differences between prompts that work at 4B and prompts that do not.

Be literal about the output. Show the exact shape you want, including an example. Small models copy structure well and infer it badly.

One job per call. A prompt that classifies, extracts and summarises will do one of the three well. Three calls to a 4B model are usually cheaper and better than one call to a 30B, and each is separately testable.

Give a decision procedure, not a description. If the message names a product and a defect, label it fault; if it names a payment or a refund, label it billing; if both, prefer billing. Ordered rules beat adjectives.

Repeat the format at the end. A single line (Reply with only the JSON object.), placed after the input rather than before it, is worth more than three sentences of instruction at the top.

Prompts do not transfer

A prompt tuned against one family will underperform on another, sometimes badly. Different instruction-tuning data taught different conventions: how strongly a system prompt binds, whether Markdown headings are natural, how the model handles a numbered list of rules, whether it prefers to explain before answering.

This has an organisational consequence people resist. Keep the prompt per model in your repository, versioned next to the model revision. When you swap models, re-tune and re-measure. Treating the prompt as portable is how a model swap gets blamed for a regression that was a prompt mismatch.

prompts/
  classify.qwen3-8b.md
  classify.gemma3-4b.md
  classify.llama31-8b.md

Where the system prompt goes when there is none

Gemma has no system role. Several other models were tuned with very few system messages and follow them weakly. Read the chat template: if system does not appear in the Jinja source, prepend your instructions to the first user message instead, and expect them to bind about as strongly.

Iterate the way you would test code

Fifteen development examples. Change one thing. Run at temperature 0. Count. Keep a log of what you tried and what it scored.

Two failure patterns to watch for. Prompt bloat: a prompt that has grown to 800 tokens through accumulated patches, half of which no longer do anything. Delete a paragraph and re-measure; you will often find the score unchanged and the cost lower. And overfitting to the fifteen: hold back another twenty-five examples you touch once at the end, or you have measured your own tuning rather than the prompt.

The most useful habit: when a prompt fails, print exactly what the model received, including the applied template. About a third of prompting problems turn out to be formatting problems wearing a prompting costume.