{"course":{"slug":"ai-for-teachers","title":"AI for Teachers","url":"https://addaly.com/learn/ai-for-teachers","level":"beginner","track":"work"},"lesson":{"slug":"peer-assessment-with-a-third-marker","title":"Peer assessment with a third marker","url":"https://addaly.com/learn/ai-for-teachers/peer-assessment-with-a-third-marker","position":33,"of":57,"minutes":8,"answer":"When students mark each other against a rubric, a model marking the same scripts becomes a third opinion that makes disagreement visible and discussable, and the disagreements, not the scores.","keyIdea":"When students mark each other against a rubric, a model marking the same scripts becomes a third opinion that makes disagreement visible and discussable, and the disagreements, not the scores, are what teach students to assess."},"body":{"format":"markdown","text":"## Why students should mark\n\nStudents who assess each other's work against a rubric learn the rubric in a way that being marked against it never teaches. They see what a 2 looks like beside a 4. They discover that \"explain the effect\" is harder to spot than they thought. And they carry that reading back into their own drafts. The research on peer assessment is consistent about this: the benefit is mostly to the assessor, not the assessed.\n\nThe difficulty has always been calibration. Two students give the same paragraph a 4 and a 2, and neither knows who is right, and the teacher does not have time to adjudicate thirty disagreements. A model as a third marker changes that, not by being right, but by being a third reading that makes the disagreement discussable.\n\n## The setup\n\nEach student marks two peers' pieces against the observable rubric from earlier in this block. Separately, the model marks all of them, with the same rubric, in one pass: codes instead of names, as always. Now every piece has three scores per criterion.\n\nShow the students where the three agree and where they do not. Agreement is uninteresting. Disagreement is the lesson.\n\n## Using the disagreement\n\nTake the check question's case: 4, 2 and 3 on \"evidence\". Put the two student markers together with the rubric and the essay and one instruction: \"Find the line in the rubric you read differently, and the place in the essay where it applies.\" Ninety seconds later they have usually found it (one counted a paraphrase as a quotation, the other did not), and they have learned the rubric's boundary in a way no explanation from the front would teach them.\n\nThe model's score is not the answer. It is a third reading that stops the argument being two-against-nothing. Sometimes the model is the outlier, and the students discover that the machine misread the essay, which is a lesson in itself and a better one than \"the computer says 3\".\n\n## What the model adds beyond a number\n\nAsk it for the reason with each score: \"For each criterion, one sentence quoting the words in the essay that placed it at this level.\" Now the three markers can compare reasons, not just numbers, and the model's sentence is a model of what a marking reason looks like. Students' own reasons get sharper after seeing a few.\n\n## Calibrating the class\n\nBefore the first real round, do a calibration round on one piece. Everyone marks it; the model marks it; the teacher marks it. Put all the scores on the board. The spread is usually wide. Discuss the criterion with the widest spread until it narrows. One calibration round cuts the disagreement in subsequent rounds by a large fraction, because the class has agreed what the words mean.\n\n## Where it belongs and where it does not\n\nPeer assessment with a third marker is for formative work: drafts, practice pieces, anything where the point is to learn. It is not for grades on a record. A peer's mark is a learning activity; a model's mark is a proxy; neither is a grade, for the reasons the marking lesson gave. Say this to the class before the first round, because some students will otherwise treat the exercise as being judged by their friends and a machine at once, and shut down.\n\n## The quiet student and the harsh one\n\nTwo patterns to watch. Some students mark everyone generously to avoid conflict; some mark harshly to seem rigorous. The model's third score exposes both: a student whose marks are consistently two levels above the model's, across every piece, is not reading the rubric. That is feedback for the marker, and it is worth giving privately.\n\n## The honest limit\n\nThe model's reading carries its own biases, and the marking lesson named them: length, fluency, conventional structure. In a peer-assessment round, that means the model will sometimes side with the student who over-marked a long fluent piece, and the class will learn the wrong boundary. Run the five-script bias check on the model with this rubric before the first round, and when its scores track length, tell the class so. \"The machine likes long answers. Watch for that.\" A class that has been told the third marker's bias can use it. A class that has not will absorb it."},"check":{"question":"Two students give a peer's essay 4/6 and 2/6 on the same criterion. The model gives it 3/6. What is the most useful thing a teacher can do with these three numbers?","options":[{"text":"Average them to 3 and record that as the mark.","correct":false,"tempting":"Turns a disagreement into a number and loses the only thing worth having, which is why two readers of the same rubric landed two levels apart."},{"text":"Take the model's score as the tie-breaker, since it applied the rubric without bias toward the student.","correct":false,"tempting":"Assumes the model is neutral, when the marking lesson showed it carries its own systematic biases; its value is as a third reading, not as a referee."},{"text":"Have the two students find the sentence in the rubric they read differently.","correct":true},{"text":"Discard the peer marks as unreliable and mark the essay herself.","correct":false,"tempting":"Loses the learning that peer assessment exists to produce; the students' disagreement is the exercise, not a failure of it."}]},"prev":{"slug":"writing-the-exam-paper","title":"Writing the exam paper","url":"https://addaly.com/learn/ai-for-teachers/writing-the-exam-paper"},"next":{"slug":"detecting-ai-writing","title":"Why detectors fail, and what to do instead","url":"https://addaly.com/learn/ai-for-teachers/detecting-ai-writing"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Peer assessment with a third marker', in AI for Teachers. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-for-teachers/peer-assessment-with-a-third-marker (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Peer assessment with a third marker","courseTitle":"AI for Teachers","module":{"index":4,"slug":"assessment-and-feedback","title":"Assessment, feedback and marking"},"answer":null,"keyIdea":"When students mark each other against a rubric, a model marking the same scripts becomes a third opinion that makes disagreement visible and discussable, and the disagreements, not the scores, are what teach students to assess.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"why-students-should-mark","runs":[{"text":"Why students should mark"}]},{"t":"p","runs":[{"text":"Students who assess each other’s work against a rubric learn the rubric in a way that being marked against it never teaches. They see what a 2 looks like beside a 4. They discover that “explain the effect” is harder to spot than they thought. And they carry that reading back into their own drafts. The research on peer assessment is consistent about this: the benefit is mostly to the assessor, not the assessed."}]},{"t":"p","runs":[{"text":"The difficulty has always been calibration. Two students give the same paragraph a 4 and a 2, and neither knows who is right, and the teacher does not have time to adjudicate thirty disagreements. A model as a third marker changes that, not by being right, but by being a third reading that makes the disagreement discussable."}]},{"t":"h2","id":"the-setup","runs":[{"text":"The setup"}]},{"t":"p","runs":[{"text":"Each student marks two peers’ pieces against the observable rubric from earlier in this block. Separately, the model marks all of them, with the same rubric, in one pass: codes instead of names, as always. Now every piece has three scores per criterion."}]},{"t":"p","runs":[{"text":"Show the students where the three agree and where they do not. Agreement is uninteresting. Disagreement is the lesson."}]},{"t":"h2","id":"using-the-disagreement","runs":[{"text":"Using the disagreement"}]},{"t":"p","runs":[{"text":"Take the check question’s case: 4, 2 and 3 on “evidence”. Put the two student markers together with the rubric and the essay and one instruction: “Find the line in the rubric you read differently, and the place in the essay where it applies.” Ninety seconds later they have usually found it (one counted a paraphrase as a quotation, the other did not), and they have learned the rubric’s boundary in a way no explanation from the front would teach them."}]},{"t":"p","runs":[{"text":"The model’s score is not the answer. It is a third reading that stops the argument being two-against-nothing. Sometimes the model is the outlier, and the students discover that the machine misread the essay, which is a lesson in itself and a better one than “the computer says 3”."}]},{"t":"h2","id":"what-the-model-adds-beyond-a-number","runs":[{"text":"What the model adds beyond a number"}]},{"t":"p","runs":[{"text":"Ask it for the reason with each score: “For each criterion, one sentence quoting the words in the essay that placed it at this level.” Now the three markers can compare reasons, not just numbers, and the model’s sentence is a model of what a marking reason looks like. Students’ own reasons get sharper after seeing a few."}]},{"t":"h2","id":"calibrating-the-class","runs":[{"text":"Calibrating the class"}]},{"t":"p","runs":[{"text":"Before the first real round, do a calibration round on one piece. Everyone marks it; the model marks it; the teacher marks it. Put all the scores on the board. The spread is usually wide. Discuss the criterion with the widest spread until it narrows. One calibration round cuts the disagreement in subsequent rounds by a large fraction, because the class has agreed what the words mean."}]},{"t":"h2","id":"where-it-belongs-and-where-it-does-not","runs":[{"text":"Where it belongs and where it does not"}]},{"t":"p","runs":[{"text":"Peer assessment with a third marker is for formative work: drafts, practice pieces, anything where the point is to learn. It is not for grades on a record. A peer’s mark is a learning activity; a model’s mark is a proxy; neither is a grade, for the reasons the marking lesson gave. Say this to the class before the first round, because some students will otherwise treat the exercise as being judged by their friends and a machine at once, and shut down."}]},{"t":"h2","id":"the-quiet-student-and-the-harsh-one","runs":[{"text":"The quiet student and the harsh one"}]},{"t":"p","runs":[{"text":"Two patterns to watch. Some students mark everyone generously to avoid conflict; some mark harshly to seem rigorous. The model’s third score exposes both: a student whose marks are consistently two levels above the model’s, across every piece, is not reading the rubric. That is feedback for the marker, and it is worth giving privately."}]},{"t":"h2","id":"the-honest-limit","runs":[{"text":"The honest limit"}],"role":"limit"},{"t":"p","runs":[{"text":"The model’s reading carries its own biases, and the marking lesson named them: length, fluency, conventional structure. In a peer-assessment round, that means the model will sometimes side with the student who over-marked a long fluent piece, and the class will learn the wrong boundary. Run the five-script bias check on the model with this rubric before the first round, and when its scores track length, tell the class so. “The machine likes long answers. Watch for that.” A class that has been told the third marker’s bias can use it. A class that has not will absorb it."}]}],"check":{"question":[{"text":"Two students give a peer’s essay 4/6 and 2/6 on the same criterion. The model gives it 3/6. What is the most useful thing a teacher can do with these three numbers?"}],"options":[{"text":[{"text":"Average them to 3 and record that as the mark."}],"correct":false,"tempting":[{"text":"Turns a disagreement into a number and loses the only thing worth having, which is why two readers of the same rubric landed two levels apart."}]},{"text":[{"text":"Take the model’s score as the tie-breaker, since it applied the rubric without bias toward the student."}],"correct":false,"tempting":[{"text":"Assumes the model is neutral, when the marking lesson showed it carries its own systematic biases; its value is as a third reading, not as a referee."}]},{"text":[{"text":"Have the two students find the sentence in the rubric they read differently."}],"correct":true,"tempting":null},{"text":[{"text":"Discard the peer marks as unreliable and mark the essay herself."}],"correct":false,"tempting":[{"text":"Loses the learning that peer assessment exists to produce; the students’ disagreement is the exercise, not a failure of it."}]}],"order":[1,3,0,2],"why":null},"prev":{"slug":"writing-the-exam-paper","title":"Writing the exam paper","minutes":9},"next":{"slug":"detecting-ai-writing","title":"Why detectors fail, and what to do instead","minutes":9}}}