What counts as evidence

Not all sources are the same weight

Students lose marks by citing a blog post as though it were a trial, and by citing a trial as though it settled a question. Knowing what a source can support is a skill markers reward heavily, and it takes about one lesson to acquire the outline of.

Roughly ascending, for a claim about whether something works:

• Anecdote. One person’s report. Establishes that something happened once, and nothing else.
• Case report or case series. Documented individual cases. Valuable for the rare and the new; cannot establish frequency or cause.
• Cross-sectional survey. A snapshot. Shows association. Cannot establish direction: did the exercise cause the mood, or the mood cause the exercise.
• Cohort study. Followed over time. Establishes temporal order, which is a real gain. Still vulnerable to confounding: the people who chose the exposure differ from those who did not, in ways you may not have measured.
• Randomised controlled trial. Allocation by chance, so on average the groups differ only in the intervention. This is what licenses causal language. Its weaknesses are its own: often small, often short, often on unrepresentative participants.
• Systematic review and meta-analysis. All the eligible studies located by a stated method and combined. The strongest form, and only as good as the studies in it. A meta-analysis of twelve small biased trials is a precise estimate of a biased effect.

The hierarchy is a guide, not a law. A well-run cohort study of 100,000 people beats a badly-run trial of 30. What matters is being able to say why.

Peer review: what it does and does not do

Peer review means two or three people in the field read the manuscript and recommended publication, usually unpaid, usually in a few hours of work. It filters out a lot of obvious rubbish. It does not verify the data, does not usually recompute the analysis, and does not guarantee replication. Plenty of peer-reviewed findings have failed to replicate: the replication crisis in psychology and several other fields is exactly this.

So peer-reviewed is a floor, not a certificate.

Preprints have not been reviewed. They are legitimate to cite in many fields, especially fast-moving ones, and you should say so: (preprint, not peer reviewed). This is normal practice, not an admission.

Predatory journals publish anything for a fee and exist to look like real journals. Check whether the journal is in the Directory of Open Access Journals, whether the editorial board is real, and whether anyone in your field has heard of it. A model will happily cite a predatory journal without comment.

The specific claims that need specific evidence

Match the claim to the design:

• X causes Y needs randomisation, or an extremely well-argued natural experiment.
• X is associated with Y needs an observational study, and the sentence must not drift into causal language in the next paragraph, which is where most essays lose the mark.
• X is common needs a representative sample, which is a much harder thing to obtain than people assume.
• X works better than Y needs a direct comparison, not two separate studies of each.

That last one is worth dwelling on. Comparing across studies (this drug had 60 per cent response in one trial, that one 45 per cent in another) is invalid, because the populations, measures and conditions differed. A model will do it happily if asked, and it reads perfectly.

Where the model helps

• Explaining a design and why it supports what it supports.
• Naming what would be needed to establish a claim you want to make. Excellent for planning research.
• Spotting causal language over correlational evidence in your own draft: find every sentence where I claim causation, and tell me what design each cited source used. A good self-check.

The honest state of things

Some of what you will cite is contested, and saying so is a strength rather than a hedge. Fields disagree; results reverse; large literatures rest on findings that later failed to replicate. Writing the evidence here is mixed, with X finding one thing in a large cohort and Y finding the opposite in a smaller trial is more accurate and better rewarded than picking one and asserting it.

A model will usually give you the consensus position smoothly and flatten the disagreement, because smooth consensus is what most text about a topic looks like. Ask specifically: what is contested about this, and who disagrees? Then go and read both sides.