Getting Real Work out of a Model · Chapter 03

How do you get this right in teaching by example?

3:01 · Narrated with a synthetic voice. Captions are available in the player.

Read the transcript

Some things are easier to show than to describe, which is why a handful of labelled examples can carry a boundary that three paragraphs of rules cannot. The catch is that examples transmit everything they have in common, not only the part you were thinking about. This figure is a real case. Four of five examples labelled refund, and refund becomes the safe guess. The same thing happens with length, with politeness, and with language. If every example is formal English, a customer writing in Hinglish gets formal English back. The audit takes one minute. Read your examples as a set, ignore the content, and finish this sentence four times: all of these are something. Whatever that something is, you have taught it. Choose examples where a careful person could have got it wrong. Three variations of where is my parcel teach nothing. The information lives where two reasonable people would disagree, and you find those by labelling fifty real items and keeping the ones where you hesitated. Then state the rule and show it, with a short reason beside each label, because an example on its own does not say what it is an example of. And check that the examples helped. Ten held-out items, three runs of each version, and count. This figure shows the two results you will actually get. Ranges that do not overlap, which is a real difference. Or ranges that overlap completely, which means your examples cost tokens and bought noise. Keep the material in the block. Ask for one finished thing. Keep the material in the block. Ask for one finished thing. Keep the material in the block. Ask for one finished thing. Keep the material in the block. Ask for one finished thing. Keep the material in the block. Ask for one finished thing. Keep the material in the block. Ask for one finished thing. Keep the material in the block. Ask for one finished thing. Keep the material in the block. Ask for one finished thing. Keep the material in the block. Ask for one finished thing. Prefer a checkable constraint over a vague hope. If the model cannot see the document, it will invent one that sounds right. Before you ship the reply, open the source yourself and match the claims. So for teaching by example, the habit is the same as the rest of this course. Put the material in the block. Ask for one finished thing. Keep constraints few and checkable. Then open a source the model did not invent before you send anything under your name. The next module continues from here.

Sources and further reading (1)