The competence inversion

The uncomfortable pattern

Here is something most regular users notice eventually and rarely say out loud. In the subject you know best, the model's output is decent but shallow, and you can see the errors. In every other subject, it seems excellent.

Both impressions cannot be trusted, and the second one is the dangerous half.

The journalist and novelist Michael Crichton described the same effect in newspapers: you read the article about your own field, see it is riddled with mistakes, turn the page, and read the article about the Middle East with full confidence. He called it Gell-Mann amnesia. It applies to AI output with more force, because the output is fluent by construction and produced on demand in exactly the volume you asked for.

So the inversion: the model is most useful where you are competent, and most trusted where you are not. Its usefulness and your ability to check it point in the same direction, and your inclination to check points the other way.

Two different jobs

Because of this, using AI inside your expertise and outside it are different activities with different rules.

Inside your field. You are using it for speed. Draft the standard section, restructure the argument, generate the boilerplate, list the cases you should consider, catch the thing you forgot. You will spot errors automatically because you have a model of the domain to notice a disagreement against. The main risk here is not error, it is homogenisation — your work drifting towards the average of what has been written before, losing the judgement that made you worth consulting.

Outside your field. You are using it for orientation, and you should be explicit that this is all it is. It is excellent at vocabulary: what is this called, what are the standard approaches, what would a professional ask me, what am I not seeing. Those outputs are checkable in a way that answers are not — you can take a term to a search engine or a professional and get somewhere.

The move that goes wrong is asking for a conclusion in a field you cannot evaluate and then acting on it. Medical, legal, tax, structural, safety. Not because the model is always wrong there, but because you have no way to tell which time it is.

Practical translations

Instead of "is this contract clause normal?", ask "what is this type of clause called, what does it typically do, and what should I ask a lawyer about it?" You have converted an unverifiable answer into a set of verifiable handles.

Instead of "what is wrong with this X-ray?", ask nothing at all, and take it to a radiologist.

Instead of "write the SQL and I will run it in production", ask for the query and read it, and if you cannot read SQL then run it against a copy and check the row counts by hand.

Instead of "summarise this 90-page report", read the report's own summary first, then use the model to interrogate the parts you did not understand, with the text in front of it. You keep the judgement about what matters.

The half-expert

One group is at unusual risk: people who know a field slightly. Enough to follow the output and find it plausible, not enough to catch the error. This is the junior professional, the enthusiastic beginner, and the manager reviewing a specialist's work.

The honest response is to name it. If you are in that position, the useful question is not "does this look right?" but "who would know, and what would I show them?" A specific claim, extracted, taken to someone competent, takes them ninety seconds to assess.

Keeping your own field sharp

The last risk is the slowest. If you always use the model for the parts of your work you find tedious, you may find in three years that the tedious part was where your fluency was maintained. This is the subject of a later lesson on deskilling, and the countermeasure begins here: keep a portion of the work unaided, deliberately, on purpose, and notice if it has become harder than it used to be.

The rule for this whole lesson fits in one line. Your confidence in an AI answer should be set by your ability to check it, not by how good the answer sounds — and those two are usually in opposite proportion.