Asking a second model

A useful habit, with one large caveat

Paste the same question, with the same material, into a different model from a different provider. Compare.

Where they agree, you have weak evidence of stability. Where they disagree, you have found the thing to check, and disagreement is the more valuable outcome: it locates the uncertainty precisely.

This costs a minute. Free tiers exist for every major model, so it costs nothing in money either.

What it is good at catching

Model-specific quirks. A habit of one system’s training that another does not share.

Genuine ambiguity. If two models read your clause differently, the clause is ambiguous, which is information about the clause and not about the models. For contract wording this is a useful signal.

Arithmetic slips, when they are not systematic.

One model’s degraded day. A routing change or an outage on one provider does not affect another.

What it does not catch: correlated errors

The caveat, and it is a real one. Different models are not independent witnesses.

They are trained on overlapping data, on much of the same public web. They share the same architecture family and similar training methods. Where the internet is confidently wrong, or where a common misconception is well represented in text, two models can be wrong in the same way for the same reason.

They also share failure modes structurally: both will lose a qualifier in the same compressed sentence, both will read an ambiguous negation the same way, both will produce a plausible figure where the document was silent.

So agreement between two models is not independent confirmation in the way that two human experts, trained differently and with different incentives, would be. Treat it as: neither of them tripped over this in an obvious way.

Doing the comparison properly

Give both the same material, in the same form. Otherwise you are comparing contexts rather than models.

Do not tell the second what the first said. “Model A said 1974: do you agree?” is the same leading question as before, aimed at a fresh model. Ask the question cold.

Compare the reasoning, not just the conclusion. Two identical conclusions reached by different routes are more reassuring than two reached identically. Two different conclusions with one clearly better-evidenced route usually resolves it.

Prefer a different provider, not a different size from the same family, since same-family models share the most. A small and a large model from one lab share training data, tokeniser and method, and will often make the same mistake with different amounts of eloquence.

The stronger version: a different kind of check

Where you can, verify against something that is not a language model at all.

• A spreadsheet or a script for arithmetic.
• The original document for a quote.
• The regulator’s own page for a rule.
• A person who knows, for a judgement.

The general principle: an independent check is one whose failure mode is different from the thing being checked. Two models fail similarly. A calculator does not fail the way a model does, which is exactly what makes it a check.

When it is worth doing

Not routinely. The cost is small but not zero, and doing it for everything trains you to skim both.

Do it when the answer surprises you, when it will be repeated to somebody else, when it goes into something you sign, or when you cannot tell from reading whether it is right. That last is the strongest trigger, and it appears again in the next lesson as the basis for the whole triage.

The version that costs nothing extra

If you already use two products (one at work, one on your phone) you have a second opinion available with no setup and no account. Most people in this position never use it, because the second tool is filed mentally as “the other one” rather than as a check.

One small habit makes it useful: when an answer will be repeated to somebody else, ask the other tool the same question cold before you repeat it. Thirty seconds, no cost, and the only two outcomes are reassurance or a specific thing to look at. There is no third outcome where you have wasted the effort.