Getting Real Work out of a Model · Chapter 02

How do you get this right in the material you give it?

3:59 · Narrated with a synthetic voice. Captions are available in the player.

Read the transcript

Context is most of the job, and one mistake dominates all the others. People describe their problem instead of showing it. My colleague sent a rude email and I want to reply firmly. Every important word there is your interpretation, and your interpretation is the thing you wanted help with. Any document task has three layers. The artefact itself. The surrounding facts, with dates. And your reading of it. The first two are what the model cannot get anywhere else. The third is worth including as a question, never as the input. So paste the email, the traceback, the clause, the header row and five real rows. And if you cannot paste, redact and paste, because the meaning is carried by the clauses rather than by the names. Once a prompt runs longer than a page, position stops being neutral. An instruction before long material helps the model read purposefully. An instruction after it is followed more reliably, because the request sits close to the answer. Those pull in opposite directions, so do both. The task in one sentence at the top, the material in the middle, the precise instruction again at the bottom, for about twenty words. This figure shows the other half of it. Material in the middle of a long context is used less well than material at either end. Newer models handle it better, and better is not solved. So put the most important document last, say where to look, and prefer the twelve-page extract to the three-hundred-page manual. Keep the material in the block. Ask for one finished thing. Keep the material in the block. Ask for one finished thing. Keep the material in the block. Ask for one finished thing. Prefer a checkable constraint over a vague hope. If the model cannot see the document, it will invent one that sounds right. Before you ship the reply, open the source yourself and match the claims. Attaching a file is not the same as pasting it, and four things can happen. The text layer is pulled out and the layout is lost. A scan goes through character recognition, where a three read as an eight looks exactly like a fact. The page is sent as an image. Or the document is chopped into chunks, indexed, and only the passages that look relevant reach the model. That last one produces a specific and nasty failure. Ask whether a contract mentions arbitration anywhere, and a negative answer might mean the arbitration chunk was never retrieved. So ask first. How many pages did you receive, what is the first line of page one, and what is the last line of the document. Keep the material in the block. Ask for one finished thing. Keep the material in the block. Ask for one finished thing. Keep the material in the block. Ask for one finished thing. Prefer a checkable constraint over a vague hope. If the model cannot see the document, it will invent one that sounds right. Before you ship the reply, open the source yourself and match the claims. So for the material you give it, the habit is the same as the rest of this course. Put the material in the block. Ask for one finished thing. Keep constraints few and checkable. Then open a source the model did not invent before you send anything under your name. The next module continues from here.

Sources and further reading (1)