Writing a rubric both can read

The rubric is the prompt

When you ask a model to assess a piece of writing, the rubric is almost the entire instruction. Everything else (the essay, the role, the tone of feedback) is secondary to what the criteria say. And most rubrics, written for human markers who share a staffroom and a training day, say things a machine cannot use.

“Demonstrates deep understanding.” “Shows strong analytical skills.” “Communicates effectively.” A colleague reads those and fills them with twenty years of shared meaning. A model reads them and finds nothing observable, so it does what the check question describes: it resolves the criterion to the surface feature that best predicted high scores in the essay-scoring data it learned from. That feature is length, and after length, vocabulary richness and sentence complexity. The marking lesson explained who those proxies punish. The rubric is where you stop it.

Observable criteria

A criterion is observable if two people, pointing at the text, could agree on whether it is met. Rewrite each vague line as a thing you could underline.

Vague: “Demonstrates understanding of the poem.”

Observable: “Quotes at least two specific phrases from the poem and, for each, explains one effect the phrase has on the reader.”

Vague: “Uses evidence well.”

Observable: “Every claim about the text is followed within two sentences by a quotation or a specific reference that supports it.”

Vague: “Well structured.”

Observable: “Each paragraph begins with a sentence that states the paragraph’s point, and the paragraphs follow the order announced in the introduction.”

Notice what happened. The criterion got longer and more specific, and it stopped rewarding length, because a two-quotation essay of 300 words meets it as fully as one of 900.

Levels with descriptors, not adjectives

A 0-5 scale needs to say what a 2 and a 4 look like, or the model (and, honestly, a tired human marker) spreads scores by feel.

Evidence (0-3)
0: No quotation or specific reference to the text.
1: One quotation or reference, not connected to a claim.
2: Two or more quotations, each connected to a claim, effect not explained.
3: Two or more quotations, each connected to a claim, with the effect on the reader explained.

Four lines. A student can read them and know what to do. A model can read them and apply them. And a parent, shown the rubric and the essay, can see why it got a 2.

Give the same rubric to both

The rubric that works for the model is the rubric that works for the student, and for the same reason: it says what to do. Hand it out with the task. Students who know that “explain the effect on the reader” is the difference between a 2 and a 3 write to it, which is the point of a rubric.

This also makes the model’s feedback legible. When it says “criterion 2, level 2: your quotations are connected to claims but the effect is not explained”, the student can find the line in the rubric and the place in their essay. Feedback against a vague rubric is just a machine’s opinion.

Testing the rubric before you use it

Take three scripts you have already marked: one strong, one middling, one weak. Give the model the rubric and the three scripts, with your marks hidden. Compare.

Two things to look for. Does it rank them the same way you did? And does it place them for the same reasons? A model that ranks correctly for the wrong reasons (the strong essay was also the longest) has not learned your rubric; it has matched your ranking by proxy. Swap in a short strong script and a long weak one and see whether the ranking survives. If not, the criterion that let length through is the one to rewrite.

What still cannot go in a rubric

Some things you mark for are real and not observable in the text: originality against what this class usually produces, the risk a student took, the improvement since October. These are judgements about the student, not the writing, and the marking lesson put them on the other side of the line. Leave them out of the rubric the model sees and keep them as your own adjustment, stated on the script. “Rubric total 11/15; +1 for the argument in paragraph 3, which nobody else attempted.”

The honest limit

An observable rubric can be met mechanically. A student who learns that two quotations with an explained effect earn a 3 will produce exactly two quotations with a formulaic explanation, and so will a model asked to write a top-scoring essay. Rubrics describe the floor of competence, not the ceiling of quality. For the pieces where quality matters (the final coursework, the piece that goes on a record) the rubric finds the floor and your reading finds the rest.