The shape of a task it is good at

Four questions, before you type

Everything in the module on why it goes wrong can be turned around into a test for whether a task is a good one. Four questions do it.

Is the answer in what I am giving it, or in its memory? A model rewriting your paragraph, summarising a document you pasted, translating your email, or finding the contradiction between two clauses you supplied is working on material in front of it. A model telling you the population of a district, the date a regulation took effect, or who wrote a paper is working from the compressed residue of its training. The first is where it is strongest; the second is where invention lives. The single biggest improvement most people can make is to move their tasks from the second column to the first.

Can I check it cheaply? A rewritten email is checked by reading it. A summary is checked against the source, if you have the source. A piece of code is checked by running it. A citation is checked only by finding the paper, which costs more than the model saved you. If checking costs more than doing, the model has not helped.

Does it matter if it is wrong one time in twenty? A first draft, a brainstorm, a name for a variable, a subject line: no. A dosage, a deadline, a contract term, a tax figure: yes. Wrong one time in twenty is roughly what to expect on specific factual detail, and it is fine for the first list and disqualifying for the second.

Is this common on the web? The model has dense support for anything millions of people have written about and thin support for anything few have. A Python function to sort a list is dense. A configuration for one obscure device is thin, and it will be answered with equal fluency.

The frontier is jagged

A 2023 experiment with several hundred consultants at a large firm gave one group a chatbot and set them realistic tasks. On tasks inside what the model could do, the assisted group produced work rated forty per cent higher in quality and finished a quarter faster. On a task deliberately chosen to sit just outside, one that needed a judgement the model’s data did not support, the assisted group was nineteen percentage points less likely to reach the right answer than the unassisted group. The authors called the boundary a jagged frontier: tasks that look equally hard to a person are not equally hard to the model, and the boundary between the two is invisible from outside.

That experiment is the honest limit of this lesson’s four questions. They tell you which side of the frontier a task is likely on. They do not draw the line, and the fluent answer will not tell you which side you are on either. The only reliable way to learn the frontier for your own work is to check the model’s output on your own tasks, for a while, and keep a note of where it failed.

Tasks that pass the four questions

Rewriting for tone, length or audience. Summarising a document you have. Turning notes into prose and prose into bullet points. Drafting a reply you will edit. Explaining a concept you can then verify against a textbook. Producing several options to choose among. Translating between major languages, for the gist. Writing code you will run, with tests. Finding what is inconsistent, missing or unclear in a document you wrote. Turning a question you cannot phrase into three questions you can search.

Tasks that fail them

Anything whose value is a specific fact you cannot check: a citation, a statistic, a date, a price, a phone number, a legal deadline. Arithmetic beyond a few digits, which the module on counting explained; use a calculator, or ask the model to write the calculation for one. Anything about events after its cut-off without a search tool attached. Anything about you, your account, your order, your file, unless the product has actually given it those things. And anything where the fluent, confident, well-structured answer is itself the danger, because you will not check it, which is the reason the last lesson of this module exists.

The habit

Before a task, ask the four questions. After it, note whether the answer was right. Within a month you will have a map of the frontier for your own work that no course could give you, and you will have moved most of your use to the side where it is reliable. That is the whole skill, and it is a skill rather than a fact, which is why it cannot simply be told.