The two arguments, and how to read them

Two conversations that keep colliding

Public argument about AI risk is really two arguments, conducted by partly different people, using the same words.

The near-term argument concerns what is happening now: discrimination, surveillance, labour conditions, misinformation, concentration of power, environmental cost, automated decisions without recourse. This is the subject of the previous eight modules. Its proponents include researchers in fairness, accountability and transparency, labour organisations, civil liberties groups and affected communities.

The long-term argument concerns whether sufficiently capable future systems could pose catastrophic or existential risk. Its proponents include a substantial number of researchers at frontier labs and academic groups working on alignment.

They are not logically opposed. A person can believe both. In practice they compete for attention, funding, regulatory bandwidth and the same scarce expert time, which is why the argument between them is sharper than the underlying disagreement warrants.

What each side says about the other

Near-term critics of long-term framing argue that speculative future risk draws attention away from documented present harm; that it is convenient for companies because it locates the danger in the future rather than the product; that it frames the solution as more capability rather than less deployment; and that it centres the concerns of a small, well-resourced group over people currently being harmed. The critique advanced in On the Dangers of Stochastic Parrots and by the researchers around it is the reference text here.

Long-term critics of near-term framing argue that present harms, though real, are the kind society has mechanisms for, and that a genuinely unprecedented risk deserves attention before rather than after; that dismissing it because the timeline is uncertain is a poor way to handle tail risks; and that the mechanisms being proposed — evaluation, interpretability, controlled deployment — help with present harms too.

Both of those are arguments made in good faith by serious people, and both contain something true.

What researchers actually think

Surveys of AI researchers find wide dispersion rather than consensus. A large 2023 survey of published machine learning researchers found a median estimate of around 5% for extremely bad long-run outcomes from advanced AI, with a large minority placing it at 10% or higher and a substantial group placing it near zero. Medians moved across successive surveys, and dispersion did not narrow.

Read that carefully. It is not "experts say it is fine" and it is not "experts say we are doomed". It is a field with no settled view, in which a nontrivial fraction of practitioners assign a small but non-negligible probability to a catastrophic outcome, and another fraction think the question is confused.

How to read any claim in this space

Five questions, which work on optimistic and pessimistic claims equally.

Is there a mechanism? Not a capability trend line, but a described causal path from here to the claimed outcome, with the steps named. Claims without a mechanism are vibes with citations.

Is there a timeline, and has this person's previous timeline passed? Predictions in this field have a public record. Checking it is a fifteen-minute exercise that reorders your priors considerably.

What would falsify it? A claim compatible with every possible observation is not a claim about the world.

Who benefits if this is believed? Applies in all directions. A company benefits from the belief that its product is so powerful it needs regulation only that company can afford, and equally from the belief that concerns are overblown. A researcher benefits from the importance of their own field. Note the incentive and then evaluate the argument anyway — incentive is a reason for care, not a refutation.

Is the confidence proportionate to the evidence? Both extremes routinely fail this. "It will certainly be fine" and "it will certainly kill us" are the two positions the evidence supports least.

Where this leaves you

A defensible position, and the one this course has taken throughout: the documented harms are documented, they are happening to identifiable people now, and the work of reducing them is available today. The long-term question is genuinely open, deserves serious people working on it, and does not license either panic or dismissal.

And the practices that make present systems safer — measurement, evaluation, containment, human oversight that actually functions, incident reporting, the ability to say no — are the same practices any future system would need. Doing them now is the part of this that requires no forecast to justify.