Nobody typed this

A request nobody wrote

At 03:14 a support assistant answers a question about a delayed refund. The request that reached the model was 9,800 tokens long. A human typed fourteen of them.

Here is where the rest came from, in the order the code appended it:

• 620 tokens of system prompt, written by a product manager in March and edited twice since.
• 1,150 tokens of tool definitions: six functions, their JSON schemas serialised into the prompt by the SDK.
• 2,900 tokens of conversation history, four earlier turns from the same customer.
• 4,100 tokens of retrieved material: nine passages pulled from a help centre by a vector search that ran forty milliseconds earlier.
• 14 tokens of what the customer actually asked.
• 1,000 tokens held back so the answer has room to exist.

No person read that request before it was sent. No person will read it afterwards unless something goes wrong. It was built by a function, from parts, most of which change on every call.

That function is the thing this course is about.

Prompting ends where assembly begins

Prompt engineering is what you do when you are the one typing. It is real work, and it has a ceiling: you can only tune the words you personally control. The moment your system starts inserting material you have never read (a customer’s earlier messages, a passage from a document somebody uploaded this morning, the output of a tool that returned 40,000 characters of JSON) the words stop being the interesting variable.

What matters then is:

• What got selected, out of everything that could have been selected.
• In what order, because order changes whether a fact is used.
• At what cost, because every token is billed, and prefill time scales with how many there are.
• From where, because material has a provenance and some of it is hostile.
• How you would prove that yesterday’s change was an improvement.

Those five questions are not answered by better wording. They are answered by code, budgets and measurement. That is why the discipline has a different name.

What is engineering about it

Call it engineering when these things become true of your context:

1. It has a budget. A number, written down, that the assembled request must not exceed, with a reserve for output.
2. It is deterministic. Given the same inputs, the assembler produces the same bytes. If it does not, you cannot debug it.
3. It is testable. You can call the assembler in a unit test without calling a model, and assert on what came out.
4. It is versioned. The whole call (instructions, retrieval settings, model id, sampling parameters) has an identifier, and a log line carries it.
5. It has failure modes you can name. Overflow, truncation, a cache miss, a poisoned passage, a tool result that swallowed the instructions.

If none of those are true, you do not have a context problem yet. You have a prompt, and prompt-level thinking will serve you fine.

The honest limit

Context engineering does not make a model correct. It changes what the model has to work with, and nothing more. A model given the right passage can still misread it; a model given a perfect budget can still invent a number.

The mechanism is worth stating plainly, because it sets the ceiling on everything else in this course. A language model produces a distribution over next tokens conditioned on the tokens in front of it. Better context shifts that distribution, sometimes dramatically. It does not add a verification step, a memory of what is true, or a refusal to guess. Those have to be built around the call, not inside the window.

So the honest claim is narrow and still worth a course: most production failures blamed on the model are failures of what the model was given. The passage was not retrieved. The instruction was 8,000 tokens above the question. The tool returned a wall of JSON and the schema at the top fell out of the window. You can fix all three without touching the model.

What you need to follow along

Everything measurable in this course is measurable for free, on a laptop, without a GPU:

• tiktoken (pip install tiktoken) counts tokens for OpenAI-family tokenisers offline.
• Hugging Face transformers gives you the exact tokeniser and chat template of any open-weights model, again offline.
• llama.cpp or Ollama runs a 3B to 8B model locally so you can run position and ordering experiments without a bill.
• Anthropic exposes a free, rate-limited token-counting endpoint (its count is a close estimate that can differ slightly from the billed figure), and OpenAI exposes POST /v1/responses/input_tokens, which returns the exact count (OpenAI’s docs do not state its price).

The paid path exists (hosted evaluation platforms, managed vector databases, observability suites) and none of it is required to learn any of this. A Python file, a JSON file of test cases and a text editor will take you through every lesson here.

Where this course goes

Nine modules, in the order the problem actually appears: what is in the window and what it costs; choosing what earns a place; the retrieval pipeline that fills it; position and ordering; cache and latency; conversation and memory; tools and agents; trust boundaries; and finally how to prove a change helped and ship it so it can be rolled back.

The first thing to do is stop calling it a prompt.