A five-minute test for whether you understood it

The illusion has a name and a measurement

Rozenblit and Keil, 2002. Ask people to rate, on a seven-point scale, how well they understand how a zip fastener works. Then ask them to write a step-by-step causal explanation. Then ask them to re-rate. The ratings drop, reliably, across zips, flush toilets, bicycles, helicopters and government policies. They called it the illusion of explanatory depth: familiarity with a thing gets mistaken for a model of it.

The drop happens at the moment of attempted explanation, not at the moment of being told the answer. That detail is what makes it useful to you. You do not need anyone to grade you. Producing the explanation is itself the test.

Why polished code triggers it harder

Processing fluency, how easily something goes down, is a cue people use for confidence, and it is a bad cue. Text in a clear font is judged more truthful than the same text in a degraded one. Consistent naming, tidy structure and no dead ends all raise fluency.

Generated code is uniformly fluent. It does not have the hesitations of human code: the variable renamed halfway through, the comment that contradicts the line below it, the leftover branch from an earlier approach. Those artefacts are friction, and friction is what used to make you stop and think. Their absence is not a sign of correctness. It is a sign that the text was produced in one pass by something with no doubts.

So you get high fluency with no relationship to your own understanding, and your confidence reads the fluency.

The test, concretely

Five minutes, no tooling. Run it on a change you are about to approve or submit.

1. Close the file. Not minimise: close.
2. Write the signature from memory. Name, parameters, return type, and what it does when it fails.
3. Name the branches. How many paths through this function, and what distinguishes them?
4. Answer three input questions without looking. Empty input. Input at the boundary: zero, one, the maximum. Input that is valid but adversarial: the duplicate, the negative, the unicode name, the clock going backwards.
5. Open the file and diff your answers against it.

If you cannot do step 2, you have read the code, not understood it. That is a fine place to be: it just means you are not yet in a position to approve it or to rely on it.

The prediction variant, which costs less

The test above is retrospective. The cheaper version runs before you generate.

Write the function signature and a comment naming the branches you expect. Then generate. Then diff your comment against the output. Two minutes.

The diff has three interesting shapes. The model handled a case you missed: that is a gap in your model of the problem, and worth thirty seconds of thought about why you missed it. You expected a case the model did not handle: that is a probable bug, and you found it before the tests ran. Or the two match, in which case you have confirmed your understanding cheaply and can move on quickly with good reason.

Signals you can check in the moment

A few questions that surface the illusion without a formal drill:

• Can you say why this approach and not the obvious alternative? If the only answer is “it works”, you have an outcome, not a reason.
• Can you predict the failure? “If the network is slow here, the user sees...”: if you cannot finish that sentence, you do not know what you are shipping.
• Could you make a deliberate one-line change that breaks it in a specific, predicted way? This is the sharpest test of all, and it is exactly what mutation testing automates.

The honest limitations

The explanatory-depth work was done on household objects and policy questions, not on code. Nobody has demonstrated the effect in a programming context with a controlled study. The transfer is an argument, not a finding.

And the drills have a real cost. Five minutes a change, on twenty changes a week, is a working hour and a half. Under a deadline this is the first thing dropped, every time, in every team. Pretending otherwise is how good practices get written down and never done. If you can only afford one, take the two-minute prediction variant, because it pays for itself by catching bugs as well as calibrating you.

You do not need to feel confused to be confused. That was always true. What changed is that the code in front of you no longer contains any of the evidence.