{"course":{"slug":"ai-for-teachers","title":"AI for Teachers","url":"https://addaly.com/learn/ai-for-teachers","level":"beginner","track":"work"},"lesson":{"slug":"marking-fairly","title":"Marking: what a machine cannot fairly judge","url":"https://addaly.com/learn/ai-for-teachers/marking-fairly","position":26,"of":57,"minutes":9,"answer":"A machine can give feedback; a grade you cannot explain to a parent is not a grade.","keyIdea":"A machine can give feedback; a grade you cannot explain to a parent is not a grade."},"body":{"format":"markdown","text":"## Where the line falls\n\nAI can help where the criterion is visible in the text and does not require knowing the student. It cannot help where the judgement is about the person: the effort, the progress, the odd but valid route to a right answer. And its failures there are not random noise. They are patterned, and the pattern falls on the same students every time.\n\n## What it does usefully\n\n- **Feedback on drafts, not grades.** \"Point to three places where the evidence does not support the claim being made.\" That is useful to a student on Tuesday in a way your marking on Friday is not.\n- **A second opinion against your own rubric.** Mark the set yourself, have it mark the same set, then read only the disagreements. The disagreements are the information.\n- **Comment banks.** Generate thirty comments for common problems, then choose. Faster than typing, and you still decide which one this student gets.\n- **Closed-form answers, with the working shown.** If it cannot show why an answer is wrong, do not accept its verdict.\n\n## Three biases, and they land on the same children\n\n**Length.** Longer answers tend to score higher. Automated scorers are shown to over-reward length, and in large-scale tests length predicts trained markers' scores even after allowing for the writer's proficiency, although one experiment found teachers in training marked long, weak answers down. The machine just applies it without tiring.\n\n**Fluency.** A student writing in their third language, making a correct and original argument in plain sentences, scores below a fluent, empty answer. This is the one to watch, because it hits precisely the students for whom fair marking matters most.\n\n**Unusual correctness.** The student who solved it a different way. A marker matching against an expected shape marks that down. You recognise it and give full credit, and then you make her show the class.\n\nAnd there is a fourth thing it cannot see: whether this is progress. Is this the first time Kwame has used paragraphs? That is often the most important fact about the piece and it exists only in your memory of him.\n\n## A protocol that catches the problem in ten minutes\n\nMark five scripts yourself. Then have the AI mark the same five. Compare.\n\nYou are not checking whether it agrees with you. You are looking for the shape of its disagreement. If it consistently rewarded the longer answers, you will see that in five. If it punished the student with weaker English and a sharper argument, you will see that too. Do this once with any new tool, and again if you change how you prompt it.\n\n## The grade you cannot explain\n\nDo not let AI assign a final grade that goes on a record without you standing behind it.\n\nThe reason is not that it is always worse than you. On some narrow tasks its agreement with trained human markers is respectable. The reason is that a parent will ask why, and \"the system gave it a 6\" is not an answer a teacher can give. There is no appeal path, no reasoning you can walk through, and nothing you can teach from. A grade you cannot defend is not a grade. It is a number.\n\n> Figure: Four answers a teacher marked the same. Bars (mark out of 10 given by the model): Long, fluent, little evidence 8; Short, correct, fluent English 7; Short, correct, second-language English 5; Correct by an unusual method 5. Four Class 10 scripts, one rubric. The teacher gave the top three a 7 and the empty fluent one a 5. The model's spread runs almost exactly with length and fluency, and the two it marked down are the second-language writer and the student who solved it a different way. Five scripts is enough to see this shape; run it once with any new tool.\n\nUse it to find things. Use your own judgement to decide things.\n\n## Before you upload thirty essays\n\nStudent work is personal data, often about children. Putting a class set with names into a consumer chatbot may breach your school's rules or your country's law, and this varies enormously: what is routine in one country is unlawful in another.\n\nThe practical minimum: strip names before anything is uploaded, ask your school what tools are approved, and if nobody knows, treat that as a no rather than a yes. This is contested and unsettled territory. Do not let a colleague's confidence stand in for a policy."},"check":{"question":"A history teacher runs thirty essays through an AI for feedback. The feedback looks good. Then she notices two essays with equally strong arguments got very different responses: the shorter one was told to develop its points, the longer one was praised. What is the honest reading?","options":[{"text":"It is responding to surface features that correlate with quality in the writing it learned from (length among them) so it can be right on average while being reliably unfair to one kind of student.","correct":true},{"text":"The longer essay probably was better; length usually reflects more effort and more detail, and the feedback simply reflected that.","correct":false,"tempting":"Length does correlate with quality often enough to feel like a fair proxy, which is exactly why the bias survives unnoticed."},{"text":"It made a one-off error; running the same essays again would give a consistent result.","correct":false,"tempting":"Treats the problem as random noise that repetition would smooth out, rather than a systematic tilt that repetition reproduces."},{"text":"This is formative feedback rather than a grade, so the unevenness does not really cost anything.","correct":false,"tempting":"Assumes feedback is harmless because it carries no mark, when repeated messages about your writing shape what a student thinks they can do."}]},"prev":{"slug":"a-unit-plan-not-just-a-lesson","title":"A unit plan, not just a lesson","url":"https://addaly.com/learn/ai-for-teachers/a-unit-plan-not-just-a-lesson"},"next":{"slug":"writing-a-rubric-both-can-read","title":"Writing a rubric both can read","url":"https://addaly.com/learn/ai-for-teachers/writing-a-rubric-both-can-read"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Marking: what a machine cannot fairly judge', in AI for Teachers. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-for-teachers/marking-fairly (Accessed: 31 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Marking: what a machine cannot fairly judge","courseTitle":"AI for Teachers","module":{"index":4,"slug":"assessment-and-feedback","title":"Assessment, feedback and marking"},"answer":null,"keyIdea":"A machine can give feedback; a grade you cannot explain to a parent is not a grade.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure 4.1","figure":{"type":"bars","unit":"mark out of 10 given by the model","title":"Four answers a teacher marked the same","series":[{"label":"Long, fluent, little evidence","value":8},{"label":"Short, correct, fluent English","value":7},{"label":"Short, correct, second-language English","value":5,"highlight":true},{"label":"Correct by an unusual method","value":5,"highlight":true}],"caption":"Four Class 10 scripts, one rubric. The teacher gave the top three a 7 and the empty fluent one a 5. The model’s spread runs almost exactly with length and fluency, and the two it marked down are the second-language writer and the student who solved it a different way. Five scripts is enough to see this shape; run it once with any new tool."},"alt":"Figure: Four answers a teacher marked the same. Bars (mark out of 10 given by the model): Long, fluent, little evidence 8; Short, correct, fluent English 7; Short, correct, second-language English 5; Correct by an unusual method 5. Four Class 10 scripts, one rubric. The teacher gave the top three a 7 and the empty fluent one a 5. The model’s spread runs almost exactly with length and fluency, and the two it marked down are the second-language writer and the student who solved it a different way. Five scripts is enough to see this shape; run it once with any new tool."}},"blocks":[{"t":"h2","id":"where-the-line-falls","runs":[{"text":"Where the line falls"}]},{"t":"p","runs":[{"text":"AI can help where the criterion is visible in the text and does not require knowing the student. It cannot help where the judgement is about the person: the effort, the progress, the odd but valid route to a right answer. And its failures there are not random noise. They are patterned, and the pattern falls on the same students every time."}]},{"t":"h2","id":"what-it-does-usefully","runs":[{"text":"What it does usefully"}]},{"t":"ul","items":[[{"text":"Feedback on drafts, not grades.","b":true},{"text":" “Point to three places where the evidence does not support the claim being made.” That is useful to a student on Tuesday in a way your marking on Friday is not."}],[{"text":"A second opinion against your own rubric.","b":true},{"text":" Mark the set yourself, have it mark the same set, then read only the disagreements. The disagreements are the information."}],[{"text":"Comment banks.","b":true},{"text":" Generate thirty comments for common problems, then choose. Faster than typing, and you still decide which one this student gets."}],[{"text":"Closed-form answers, with the working shown.","b":true},{"text":" If it cannot show why an answer is wrong, do not accept its verdict."}]]},{"t":"h2","id":"three-biases-and-they-land-on-the-same-children","runs":[{"text":"Three biases, and they land on the same children"}]},{"t":"p","runs":[{"text":"Length.","b":true},{"text":" Longer answers tend to score higher. Automated scorers are shown to over-reward length, and in large-scale tests length predicts trained markers’ scores even after allowing for the writer’s proficiency, although one experiment found teachers in training marked long, weak answers down. The machine just applies it without tiring."}]},{"t":"p","runs":[{"text":"Fluency.","b":true},{"text":" A student writing in their third language, making a correct and original argument in plain sentences, scores below a fluent, empty answer. This is the one to watch, because it hits precisely the students for whom fair marking matters most."}]},{"t":"p","runs":[{"text":"Unusual correctness.","b":true},{"text":" The student who solved it a different way. A marker matching against an expected shape marks that down. You recognise it and give full credit, and then you make her show the class."}]},{"t":"p","runs":[{"text":"And there is a fourth thing it cannot see: whether this is progress. Is this the first time Kwame has used paragraphs? That is often the most important fact about the piece and it exists only in your memory of him."}]},{"t":"h2","id":"a-protocol-that-catches-the-problem-in-ten-minutes","runs":[{"text":"A protocol that catches the problem in ten minutes"}]},{"t":"p","runs":[{"text":"Mark five scripts yourself. Then have the AI mark the same five. Compare."}]},{"t":"p","runs":[{"text":"You are not checking whether it agrees with you. You are looking for the shape of its disagreement. If it consistently rewarded the longer answers, you will see that in five. If it punished the student with weaker English and a sharper argument, you will see that too. Do this once with any new tool, and again if you change how you prompt it."}]},{"t":"h2","id":"the-grade-you-cannot-explain","runs":[{"text":"The grade you cannot explain"}]},{"t":"p","runs":[{"text":"Do not let AI assign a final grade that goes on a record without you standing behind it."}]},{"t":"p","runs":[{"text":"The reason is not that it is always worse than you. On some narrow tasks its agreement with trained human markers is respectable. The reason is that a parent will ask why, and “the system gave it a 6” is not an answer a teacher can give. There is no appeal path, no reasoning you can walk through, and nothing you can teach from. A grade you cannot defend is not a grade. It is a number."}]},{"t":"p","runs":[{"text":"Use it to find things. Use your own judgement to decide things."}]},{"t":"h2","id":"before-you-upload-thirty-essays","runs":[{"text":"Before you upload thirty essays"}]},{"t":"p","runs":[{"text":"Student work is personal data, often about children. Putting a class set with names into a consumer chatbot may breach your school’s rules or your country’s law, and this varies enormously: what is routine in one country is unlawful in another."}]},{"t":"p","runs":[{"text":"The practical minimum: strip names before anything is uploaded, ask your school what tools are approved, and if nobody knows, treat that as a no rather than a yes. This is contested and unsettled territory. Do not let a colleague’s confidence stand in for a policy."}]}],"check":{"question":[{"text":"A history teacher runs thirty essays through an AI for feedback. The feedback looks good. Then she notices two essays with equally strong arguments got very different responses: the shorter one was told to develop its points, the longer one was praised. What is the honest reading?"}],"options":[{"text":[{"text":"It is responding to surface features that correlate with quality in the writing it learned from (length among them) so it can be right on average while being reliably unfair to one kind of student."}],"correct":true,"tempting":null},{"text":[{"text":"The longer essay probably was better; length usually reflects more effort and more detail, and the feedback simply reflected that."}],"correct":false,"tempting":[{"text":"Length does correlate with quality often enough to feel like a fair proxy, which is exactly why the bias survives unnoticed."}]},{"text":[{"text":"It made a one-off error; running the same essays again would give a consistent result."}],"correct":false,"tempting":[{"text":"Treats the problem as random noise that repetition would smooth out, rather than a systematic tilt that repetition reproduces."}]},{"text":[{"text":"This is formative feedback rather than a grade, so the unevenness does not really cost anything."}],"correct":false,"tempting":[{"text":"Assumes feedback is harmless because it carries no mark, when repeated messages about your writing shape what a student thinks they can do."}]}],"order":[3,2,0,1],"why":null},"prev":{"slug":"a-unit-plan-not-just-a-lesson","title":"A unit plan, not just a lesson","minutes":8},"next":{"slug":"writing-a-rubric-both-can-read","title":"Writing a rubric both can read","minutes":9}}}