{"course":{"slug":"ai-for-teachers","title":"AI for Teachers","url":"https://addaly.com/learn/ai-for-teachers","level":"beginner","track":"work"},"lesson":{"slug":"writing-a-rubric-both-can-read","title":"Writing a rubric both can read","url":"https://addaly.com/learn/ai-for-teachers/writing-a-rubric-both-can-read","position":27,"of":57,"minutes":9,"answer":"A rubric criterion must be observable in the text (\"cites a specific line and explains its effect\" rather than \"shows understanding\") because a vague criterion makes the model fall back on length and fluency.","keyIdea":"A rubric criterion must be observable in the text (\"cites a specific line and explains its effect\" rather than \"shows understanding\") because a vague criterion makes the model fall back on length and fluency, which are the biases that land on your weakest writers."},"body":{"format":"markdown","text":"## The rubric is the prompt\n\nWhen you ask a model to assess a piece of writing, the rubric is almost the entire instruction. Everything else (the essay, the role, the tone of feedback) is secondary to what the criteria say. And most rubrics, written for human markers who share a staffroom and a training day, say things a machine cannot use.\n\n\"Demonstrates deep understanding.\" \"Shows strong analytical skills.\" \"Communicates effectively.\" A colleague reads those and fills them with twenty years of shared meaning. A model reads them and finds nothing observable, so it does what the check question describes: it resolves the criterion to the surface feature that best predicted high scores in the essay-scoring data it learned from. That feature is length, and after length, vocabulary richness and sentence complexity. The marking lesson explained who those proxies punish. The rubric is where you stop it.\n\n## Observable criteria\n\nA criterion is observable if two people, pointing at the text, could agree on whether it is met. Rewrite each vague line as a thing you could underline.\n\n> Figure: A criterion two people could point at. Vague: the model resolves it to length: Demonstrates deep understanding; Shows strong analytical skills; Uses evidence well; Well structured; Communicates effectively. Observable: you could underline it: Quotes two phrases and explains one effect of each; Every claim is followed within two sentences by a quotation; Each paragraph opens with a sentence stating its point; Paragraphs follow the order announced in the introduction. A vague criterion gives the model nothing observable, so it falls back on the surface features that predicted high scores in essay-marking data: length first, then vocabulary and sentence complexity. The observable version is longer and more specific, and it stops rewarding length, because a 300-word answer can meet it as fully as a 900-word one.\n\nVague: \"Demonstrates understanding of the poem.\"\n\nObservable: \"Quotes at least two specific phrases from the poem and, for each, explains one effect the phrase has on the reader.\"\n\nVague: \"Uses evidence well.\"\n\nObservable: \"Every claim about the text is followed within two sentences by a quotation or a specific reference that supports it.\"\n\nVague: \"Well structured.\"\n\nObservable: \"Each paragraph begins with a sentence that states the paragraph's point, and the paragraphs follow the order announced in the introduction.\"\n\nNotice what happened. The criterion got longer and more specific, and it stopped rewarding length, because a two-quotation essay of 300 words meets it as fully as one of 900.\n\n## Levels with descriptors, not adjectives\n\nA 0-5 scale needs to say what a 2 and a 4 look like, or the model (and, honestly, a tired human marker) spreads scores by feel.\n\n```text\nEvidence (0-3)\n0: No quotation or specific reference to the text.\n1: One quotation or reference, not connected to a claim.\n2: Two or more quotations, each connected to a claim, effect not explained.\n3: Two or more quotations, each connected to a claim, with the effect on the reader explained.\n```\n\nFour lines. A student can read them and know what to do. A model can read them and apply them. And a parent, shown the rubric and the essay, can see why it got a 2.\n\n## Give the same rubric to both\n\nThe rubric that works for the model is the rubric that works for the student, and for the same reason: it says what to do. Hand it out with the task. Students who know that \"explain the effect on the reader\" is the difference between a 2 and a 3 write to it, which is the point of a rubric.\n\nThis also makes the model's feedback legible. When it says \"criterion 2, level 2: your quotations are connected to claims but the effect is not explained\", the student can find the line in the rubric and the place in their essay. Feedback against a vague rubric is just a machine's opinion.\n\n## Testing the rubric before you use it\n\nTake three scripts you have already marked: one strong, one middling, one weak. Give the model the rubric and the three scripts, with your marks hidden. Compare.\n\nTwo things to look for. Does it rank them the same way you did? And does it place them for the same reasons? A model that ranks correctly for the wrong reasons (the strong essay was also the longest) has not learned your rubric; it has matched your ranking by proxy. Swap in a short strong script and a long weak one and see whether the ranking survives. If not, the criterion that let length through is the one to rewrite.\n\n## What still cannot go in a rubric\n\nSome things you mark for are real and not observable in the text: originality against what this class usually produces, the risk a student took, the improvement since October. These are judgements about the student, not the writing, and the marking lesson put them on the other side of the line. Leave them out of the rubric the model sees and keep them as your own adjustment, stated on the script. \"Rubric total 11/15; +1 for the argument in paragraph 3, which nobody else attempted.\"\n\n## The honest limit\n\nAn observable rubric can be met mechanically. A student who learns that two quotations with an explained effect earn a 3 will produce exactly two quotations with a formulaic explanation, and so will a model asked to write a top-scoring essay. Rubrics describe the floor of competence, not the ceiling of quality. For the pieces where quality matters (the final coursework, the piece that goes on a record) the rubric finds the floor and your reading finds the rest."},"check":{"question":"A teacher gives an AI her rubric, which includes \"demonstrates deep understanding of the poem (0-5)\". The scores it produces correlate almost perfectly with essay length. What is the mechanism?","options":[{"text":"The model equates understanding with effort, and the length of an essay is the only measure of effort available to it.","correct":false,"tempting":"Attributes a value judgement; the model has no notion of effort, it resolves an unobservable criterion to whatever surface feature best predicts high scores in the text it learned from."},{"text":"The criterion names nothing observable, so the model scored the nearest visible proxy, which is length.","correct":true},{"text":"The 0-5 scale is too coarse for this criterion, and a 0-10 scale would have separated understanding from length.","correct":false,"tempting":"Scale granularity does not create an observable criterion; a finer scale on a vague criterion produces finer-grained length scoring."},{"text":"The model did not read the poem itself, so it could not judge understanding of it and fell back on length as a default.","correct":false,"tempting":"Reading the poem would not help; the criterion still gives no description of what understanding looks like on the page."}]},"prev":{"slug":"marking-fairly","title":"Marking: what a machine cannot fairly judge","url":"https://addaly.com/learn/ai-for-teachers/marking-fairly"},"next":{"slug":"feedback-students-actually-use","title":"Feedback students actually use","url":"https://addaly.com/learn/ai-for-teachers/feedback-students-actually-use"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Writing a rubric both can read', in AI for Teachers. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-for-teachers/writing-a-rubric-both-can-read (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Writing a rubric both can read","courseTitle":"AI for Teachers","module":{"index":4,"slug":"assessment-and-feedback","title":"Assessment, feedback and marking"},"answer":null,"keyIdea":"A rubric criterion must be observable in the text (“cites a specific line and explains its effect” rather than “shows understanding”) because a vague criterion makes the model fall back on length and fluency, which are the biases that land on your weakest writers.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"left":{"title":"Vague: the model resolves it to length","items":["Demonstrates deep understanding","Shows strong analytical skills","Uses evidence well","Well structured","Communicates effectively"]},"type":"compare","right":{"title":"Observable: you could underline it","items":["Quotes two phrases and explains one effect of each","Every claim is followed within two sentences by a quotation","Each paragraph opens with a sentence stating its point","Paragraphs follow the order announced in the introduction"]},"title":"A criterion two people could point at","caption":"A vague criterion gives the model nothing observable, so it falls back on the surface features that predicted high scores in essay-marking data: length first, then vocabulary and sentence complexity. The observable version is longer and more specific, and it stops rewarding length, because a 300-word answer can meet it as fully as a 900-word one."},"alt":"Figure: A criterion two people could point at. Vague: the model resolves it to length: Demonstrates deep understanding; Shows strong analytical skills; Uses evidence well; Well structured; Communicates effectively. Observable: you could underline it: Quotes two phrases and explains one effect of each; Every claim is followed within two sentences by a quotation; Each paragraph opens with a sentence stating its point; Paragraphs follow the order announced in the introduction. A vague criterion gives the model nothing observable, so it falls back on the surface features that predicted high scores in essay-marking data: length first, then vocabulary and sentence complexity. The observable version is longer and more specific, and it stops rewarding length, because a 300-word answer can meet it as fully as a 900-word one."}},"blocks":[{"t":"h2","id":"the-rubric-is-the-prompt","runs":[{"text":"The rubric is the prompt"}]},{"t":"p","runs":[{"text":"When you ask a model to assess a piece of writing, the rubric is almost the entire instruction. Everything else (the essay, the role, the tone of feedback) is secondary to what the criteria say. And most rubrics, written for human markers who share a staffroom and a training day, say things a machine cannot use."}]},{"t":"p","runs":[{"text":"“Demonstrates deep understanding.” “Shows strong analytical skills.” “Communicates effectively.” A colleague reads those and fills them with twenty years of shared meaning. A model reads them and finds nothing observable, so it does what the check question describes: it resolves the criterion to the surface feature that best predicted high scores in the essay-scoring data it learned from. That feature is length, and after length, vocabulary richness and sentence complexity. The marking lesson explained who those proxies punish. The rubric is where you stop it."}]},{"t":"h2","id":"observable-criteria","runs":[{"text":"Observable criteria"}]},{"t":"p","runs":[{"text":"A criterion is observable if two people, pointing at the text, could agree on whether it is met. Rewrite each vague line as a thing you could underline."}]},{"t":"p","runs":[{"text":"Vague: “Demonstrates understanding of the poem.”"}]},{"t":"p","runs":[{"text":"Observable: “Quotes at least two specific phrases from the poem and, for each, explains one effect the phrase has on the reader.”"}]},{"t":"p","runs":[{"text":"Vague: “Uses evidence well.”"}]},{"t":"p","runs":[{"text":"Observable: “Every claim about the text is followed within two sentences by a quotation or a specific reference that supports it.”"}]},{"t":"p","runs":[{"text":"Vague: “Well structured.”"}]},{"t":"p","runs":[{"text":"Observable: “Each paragraph begins with a sentence that states the paragraph’s point, and the paragraphs follow the order announced in the introduction.”"}]},{"t":"p","runs":[{"text":"Notice what happened. The criterion got longer and more specific, and it stopped rewarding length, because a two-quotation essay of 300 words meets it as fully as one of 900."}]},{"t":"h2","id":"levels-with-descriptors-not-adjectives","runs":[{"text":"Levels with descriptors, not adjectives"}]},{"t":"p","runs":[{"text":"A 0-5 scale needs to say what a 2 and a 4 look like, or the model (and, honestly, a tired human marker) spreads scores by feel."}]},{"t":"code","lang":"text","label":"Text","text":"Evidence (0-3)\n0: No quotation or specific reference to the text.\n1: One quotation or reference, not connected to a claim.\n2: Two or more quotations, each connected to a claim, effect not explained.\n3: Two or more quotations, each connected to a claim, with the effect on the reader explained."},{"t":"p","runs":[{"text":"Four lines. A student can read them and know what to do. A model can read them and apply them. And a parent, shown the rubric and the essay, can see why it got a 2."}]},{"t":"h2","id":"give-the-same-rubric-to-both","runs":[{"text":"Give the same rubric to both"}]},{"t":"p","runs":[{"text":"The rubric that works for the model is the rubric that works for the student, and for the same reason: it says what to do. Hand it out with the task. Students who know that “explain the effect on the reader” is the difference between a 2 and a 3 write to it, which is the point of a rubric."}]},{"t":"p","runs":[{"text":"This also makes the model’s feedback legible. When it says “criterion 2, level 2: your quotations are connected to claims but the effect is not explained”, the student can find the line in the rubric and the place in their essay. Feedback against a vague rubric is just a machine’s opinion."}]},{"t":"h2","id":"testing-the-rubric-before-you-use-it","runs":[{"text":"Testing the rubric before you use it"}]},{"t":"p","runs":[{"text":"Take three scripts you have already marked: one strong, one middling, one weak. Give the model the rubric and the three scripts, with your marks hidden. Compare."}]},{"t":"p","runs":[{"text":"Two things to look for. Does it rank them the same way you did? And does it place them for the same reasons? A model that ranks correctly for the wrong reasons (the strong essay was also the longest) has not learned your rubric; it has matched your ranking by proxy. Swap in a short strong script and a long weak one and see whether the ranking survives. If not, the criterion that let length through is the one to rewrite."}]},{"t":"h2","id":"what-still-cannot-go-in-a-rubric","runs":[{"text":"What still cannot go in a rubric"}]},{"t":"p","runs":[{"text":"Some things you mark for are real and not observable in the text: originality against what this class usually produces, the risk a student took, the improvement since October. These are judgements about the student, not the writing, and the marking lesson put them on the other side of the line. Leave them out of the rubric the model sees and keep them as your own adjustment, stated on the script. “Rubric total 11/15; +1 for the argument in paragraph 3, which nobody else attempted.”"}]},{"t":"h2","id":"the-honest-limit","runs":[{"text":"The honest limit"}],"role":"limit"},{"t":"p","runs":[{"text":"An observable rubric can be met mechanically. A student who learns that two quotations with an explained effect earn a 3 will produce exactly two quotations with a formulaic explanation, and so will a model asked to write a top-scoring essay. Rubrics describe the floor of competence, not the ceiling of quality. For the pieces where quality matters (the final coursework, the piece that goes on a record) the rubric finds the floor and your reading finds the rest."}]}],"check":{"question":[{"text":"A teacher gives an AI her rubric, which includes “demonstrates deep understanding of the poem (0-5)”. The scores it produces correlate almost perfectly with essay length. What is the mechanism?"}],"options":[{"text":[{"text":"The model equates understanding with effort, and the length of an essay is the only measure of effort available to it."}],"correct":false,"tempting":[{"text":"Attributes a value judgement; the model has no notion of effort, it resolves an unobservable criterion to whatever surface feature best predicts high scores in the text it learned from."}]},{"text":[{"text":"The criterion names nothing observable, so the model scored the nearest visible proxy, which is length."}],"correct":true,"tempting":null},{"text":[{"text":"The 0-5 scale is too coarse for this criterion, and a 0-10 scale would have separated understanding from length."}],"correct":false,"tempting":[{"text":"Scale granularity does not create an observable criterion; a finer scale on a vague criterion produces finer-grained length scoring."}]},{"text":[{"text":"The model did not read the poem itself, so it could not judge understanding of it and fell back on length as a default."}],"correct":false,"tempting":[{"text":"Reading the poem would not help; the criterion still gives no description of what understanding looks like on the page."}]}],"order":[0,1,3,2],"why":null},"prev":{"slug":"marking-fairly","title":"Marking: what a machine cannot fairly judge","minutes":9},"next":{"slug":"feedback-students-actually-use","title":"Feedback students actually use","minutes":8}}}