Peer assessment with a third marker

Why students should mark

Students who assess each other’s work against a rubric learn the rubric in a way that being marked against it never teaches. They see what a 2 looks like beside a 4. They discover that “explain the effect” is harder to spot than they thought. And they carry that reading back into their own drafts. The research on peer assessment is consistent about this: the benefit is mostly to the assessor, not the assessed.

The difficulty has always been calibration. Two students give the same paragraph a 4 and a 2, and neither knows who is right, and the teacher does not have time to adjudicate thirty disagreements. A model as a third marker changes that, not by being right, but by being a third reading that makes the disagreement discussable.

The setup

Each student marks two peers’ pieces against the observable rubric from earlier in this block. Separately, the model marks all of them, with the same rubric, in one pass: codes instead of names, as always. Now every piece has three scores per criterion.

Show the students where the three agree and where they do not. Agreement is uninteresting. Disagreement is the lesson.

Using the disagreement

Take the check question’s case: 4, 2 and 3 on “evidence”. Put the two student markers together with the rubric and the essay and one instruction: “Find the line in the rubric you read differently, and the place in the essay where it applies.” Ninety seconds later they have usually found it (one counted a paraphrase as a quotation, the other did not), and they have learned the rubric’s boundary in a way no explanation from the front would teach them.

The model’s score is not the answer. It is a third reading that stops the argument being two-against-nothing. Sometimes the model is the outlier, and the students discover that the machine misread the essay, which is a lesson in itself and a better one than “the computer says 3”.

What the model adds beyond a number

Ask it for the reason with each score: “For each criterion, one sentence quoting the words in the essay that placed it at this level.” Now the three markers can compare reasons, not just numbers, and the model’s sentence is a model of what a marking reason looks like. Students’ own reasons get sharper after seeing a few.

Calibrating the class

Before the first real round, do a calibration round on one piece. Everyone marks it; the model marks it; the teacher marks it. Put all the scores on the board. The spread is usually wide. Discuss the criterion with the widest spread until it narrows. One calibration round cuts the disagreement in subsequent rounds by a large fraction, because the class has agreed what the words mean.

Where it belongs and where it does not

Peer assessment with a third marker is for formative work: drafts, practice pieces, anything where the point is to learn. It is not for grades on a record. A peer’s mark is a learning activity; a model’s mark is a proxy; neither is a grade, for the reasons the marking lesson gave. Say this to the class before the first round, because some students will otherwise treat the exercise as being judged by their friends and a machine at once, and shut down.

The quiet student and the harsh one

Two patterns to watch. Some students mark everyone generously to avoid conflict; some mark harshly to seem rigorous. The model’s third score exposes both: a student whose marks are consistently two levels above the model’s, across every piece, is not reading the rubric. That is feedback for the marker, and it is worth giving privately.

The honest limit

The model’s reading carries its own biases, and the marking lesson named them: length, fluency, conventional structure. In a peer-assessment round, that means the model will sometimes side with the student who over-marked a long fluent piece, and the class will learn the wrong boundary. Run the five-script bias check on the model with this rubric before the first round, and when its scores track length, tell the class so. “The machine likes long answers. Watch for that.” A class that has been told the third marker’s bias can use it. A class that has not will absorb it.