{"course":{"slug":"ai-for-students","title":"AI for Students","url":"https://addaly.com/learn/ai-for-students","level":"beginner","track":"work"},"lesson":{"slug":"how-detectors-work","title":"How detectors work, and why they cannot","url":"https://addaly.com/learn/ai-for-students/how-detectors-work","position":59,"of":73,"minutes":9,"answer":"Detectors measure perplexity and burstiness (how typical the writing is) so they systematically misclassify second-language and formulaic writers.","keyIdea":"Detectors measure perplexity and burstiness (how typical the writing is) so they systematically misclassify second-language and formulaic writers, and the base rate means roughly half of flagged essays can be innocent even with generous accuracy assumptions."},"body":{"format":"markdown","text":"## What they actually measure\n\nAI text detectors do not identify the origin of a document. They measure statistical properties of the text and compare them against what generated text tends to look like. The two most common signals:\n\n**Perplexity.** Roughly, how surprised a language model is by each word given the previous ones. Generated text tends to have low perplexity, because it was produced by picking likely continuations. So a detector infers: unsurprising text, probably generated.\n\n**Burstiness.** Variation in sentence length and complexity across a document. Human writing tends to vary more; generated writing tends to be more even.\n\nNow read those again with a student in mind. What kind of human writing has low perplexity and low burstiness? Careful, formal, well-edited prose in a conventional structure. A student writing to a rubric in a second language, using standard academic phrasing, produces exactly that profile.\n\n**The detector is measuring how typical your writing is, not who produced it.** That is not a flaw in a particular product. It is what the measurement is.\n\n## The base rate problem\n\nSuppose a detector is 95 per cent accurate in both directions (better than the evidence supports) and 5 per cent of a 1,000-essay cohort used AI in a prohibited way.\n\n- 50 essays are AI-written. 47 or 48 are flagged.\n- 950 essays are human. 5 per cent of them, about 47, are also flagged.\n\nSo about half of all flagged essays are innocent. That is with generous accuracy assumptions and a fairly high prevalence. Lower the prevalence or the accuracy and it gets worse fast.\n\n> Figure: A detector that is 95 per cent accurate, on a thousand essays. A two by two grid, rows Used AI (50) and Did not (950), columns Flagged and Not flagged: Used AI (50) and Flagged: 47; Used AI (50) and Not flagged: 3; Did not (950) and Flagged: 47; Did not (950) and Not flagged: 903. Take the generous case and assume one essay in twenty broke the rule. Forty-seven of the fifty are caught, and so are forty-seven of the nine hundred and fifty who did nothing. About half of everyone flagged is innocent, and lowering either the accuracy or the prevalence makes it worse quickly. This is the arithmetic that governs medical screening, and a screening result is not a verdict.\n\nThis is the same arithmetic that governs medical screening, and it is why a positive result on a rare condition often means very little on its own. A detector score is a screening result, and treating it as a verdict is a category error with a name.\n\n## The unequal harm\n\nThe false positives are not randomly distributed, and this is the part that should trouble everyone. Published evaluations have found detectors flagging writing by non-native English speakers at substantially higher rates than native speakers: in one widely-reported study, a majority of essays by non-native speakers were misclassified as AI-generated, while nearly all native-speaker essays were classified correctly.\n\nThe mechanism explains it exactly. Learners of a language use more common words, more standard constructions, less varied sentence structure. That is low perplexity and low burstiness. The detector is measuring the language proficiency and reporting it as authorship.\n\nThe same logic exposes autistic students who write in a consistent register, students taught to write formulaically, and anyone using an editing tool, including the spelling and grammar checkers that have been permitted for decades.\n\n## What the vendors and institutions now say\n\nSeveral major providers state that their scores are indicative rather than conclusive and should not be the sole basis of an allegation. OpenAI withdrew its own classifier in 2023, citing low accuracy. A number of universities have disabled detector tools in their submission systems for exactly these reasons, and others have adopted policies stating that a detector score alone cannot support a finding.\n\nSo if a score is being used against you as though it were proof, that is not only unfair: it is usually contrary to the tool's own documentation, and saying so calmly is a legitimate part of a response.\n\n## What this does not mean\n\nIt does not mean nobody can tell. Markers who know a subject and have read your previous work recognise a change in register, an argument that does not connect to the seminar you were in, a bibliography with fabricated entries, and an answer that cannot be defended in conversation. Those are evidence, and they are how most cases are actually made and how most are correctly made.\n\nIt also does not mean generated submissions go undetected in practice. The most reliable methods are the human ones: a viva, a question about your own paragraph, an in-person written task compared against your coursework.\n\n## What to do about it\n\nYou cannot control whether a detector flags you. You can control whether you have evidence, and that is the whole of the next lesson.\n\n- Write in a tool with version history and leave it on.\n- Keep drafts, notes, and the reading you did.\n- Keep the unaided-check log from Module 3.\n- If your writing is naturally formulaic, or you are writing in a second language, be especially deliberate about this. The system's error falls on you disproportionately, and the only available protection is a record.\n\nThat is an unjust distribution of effort. It is also, currently, the situation."},"check":{"question":"A detector is 95 per cent accurate in both directions and 5 per cent of a 1,000-essay cohort used AI. Why does a flag tell a marker so little?","options":[{"text":"Because 95 per cent accuracy is measured on benchmark datasets that do not resemble student coursework.","correct":false,"tempting":"Raises a real external-validity concern that would make things worse, but the arithmetic already fails at the stated accuracy."},{"text":"Because a 5 per cent prevalence estimate is itself a guess, and without knowing the true rate no inference can be drawn from a positive result.","correct":false,"tempting":"Uncertainty in the prevalence widens the range but does not remove the effect, which holds across every plausible value."},{"text":"Because accuracy figures describe the tool's performance on average and cannot be applied to an individual essay at all.","correct":false,"tempting":"Overstates into a claim that no probabilistic evidence is ever informative about a case; the point is the size of the resulting probability, not its inapplicability."},{"text":"Because the innocent group is so much larger that its false positives roughly equal the true positives.","correct":true}]},"prev":{"slug":"writing-a-disclosure","title":"Writing a disclosure that helps you","url":"https://addaly.com/learn/ai-for-students/writing-a-disclosure"},"next":{"slug":"the-process-record","title":"The process record","url":"https://addaly.com/learn/ai-for-students/the-process-record"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'How detectors work, and why they cannot', in AI for Students. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-for-students/how-detectors-work (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"How detectors work, and why they cannot","courseTitle":"AI for Students","module":{"index":7,"slug":"rules-honesty-record","title":"Rules, honesty and the record"},"answer":null,"keyIdea":"Detectors measure perplexity and burstiness (how typical the writing is) so they systematically misclassify second-language and formulaic writers, and the base rate means roughly half of flagged essays can be innocent even with generous accuracy assumptions.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"cols":["Flagged","Not flagged"],"rows":["Used AI (50)","Did not (950)"],"type":"matrix","cells":[["47","3"],["47","903"]],"title":"A detector that is 95 per cent accurate, on a thousand essays","caption":"Take the generous case and assume one essay in twenty broke the rule. Forty-seven of the fifty are caught, and so are forty-seven of the nine hundred and fifty who did nothing. About half of everyone flagged is innocent, and lowering either the accuracy or the prevalence makes it worse quickly. This is the arithmetic that governs medical screening, and a screening result is not a verdict.","cellLabels":[["correctly caught","missed entirely"],["wrongly accused","cleared"]]},"alt":"Figure: A detector that is 95 per cent accurate, on a thousand essays. A two by two grid, rows Used AI (50) and Did not (950), columns Flagged and Not flagged: Used AI (50) and Flagged: 47; Used AI (50) and Not flagged: 3; Did not (950) and Flagged: 47; Did not (950) and Not flagged: 903. Take the generous case and assume one essay in twenty broke the rule. Forty-seven of the fifty are caught, and so are forty-seven of the nine hundred and fifty who did nothing. About half of everyone flagged is innocent, and lowering either the accuracy or the prevalence makes it worse quickly. This is the arithmetic that governs medical screening, and a screening result is not a verdict."}},"blocks":[{"t":"h2","id":"what-they-actually-measure","runs":[{"text":"What they actually measure"}]},{"t":"p","runs":[{"text":"AI text detectors do not identify the origin of a document. They measure statistical properties of the text and compare them against what generated text tends to look like. The two most common signals:"}]},{"t":"p","runs":[{"text":"Perplexity.","b":true},{"text":" Roughly, how surprised a language model is by each word given the previous ones. Generated text tends to have low perplexity, because it was produced by picking likely continuations. So a detector infers: unsurprising text, probably generated."}]},{"t":"p","runs":[{"text":"Burstiness.","b":true},{"text":" Variation in sentence length and complexity across a document. Human writing tends to vary more; generated writing tends to be more even."}]},{"t":"p","runs":[{"text":"Now read those again with a student in mind. What kind of human writing has low perplexity and low burstiness? Careful, formal, well-edited prose in a conventional structure. A student writing to a rubric in a second language, using standard academic phrasing, produces exactly that profile."}]},{"t":"p","runs":[{"text":"The detector is measuring how typical your writing is, not who produced it.","b":true},{"text":" That is not a flaw in a particular product. It is what the measurement is."}]},{"t":"h2","id":"the-base-rate-problem","runs":[{"text":"The base rate problem"}]},{"t":"p","runs":[{"text":"Suppose a detector is 95 per cent accurate in both directions (better than the evidence supports) and 5 per cent of a 1,000-essay cohort used AI in a prohibited way."}]},{"t":"ul","items":[[{"text":"50 essays are AI-written. 47 or 48 are flagged."}],[{"text":"950 essays are human. 5 per cent of them, about 47, are also flagged."}]]},{"t":"p","runs":[{"text":"So about half of all flagged essays are innocent. That is with generous accuracy assumptions and a fairly high prevalence. Lower the prevalence or the accuracy and it gets worse fast."}]},{"t":"p","runs":[{"text":"This is the same arithmetic that governs medical screening, and it is why a positive result on a rare condition often means very little on its own. A detector score is a screening result, and treating it as a verdict is a category error with a name."}]},{"t":"h2","id":"the-unequal-harm","runs":[{"text":"The unequal harm"}]},{"t":"p","runs":[{"text":"The false positives are not randomly distributed, and this is the part that should trouble everyone. Published evaluations have found detectors flagging writing by non-native English speakers at substantially higher rates than native speakers: in one widely-reported study, a majority of essays by non-native speakers were misclassified as AI-generated, while nearly all native-speaker essays were classified correctly."}]},{"t":"p","runs":[{"text":"The mechanism explains it exactly. Learners of a language use more common words, more standard constructions, less varied sentence structure. That is low perplexity and low burstiness. The detector is measuring the language proficiency and reporting it as authorship."}]},{"t":"p","runs":[{"text":"The same logic exposes autistic students who write in a consistent register, students taught to write formulaically, and anyone using an editing tool, including the spelling and grammar checkers that have been permitted for decades."}]},{"t":"h2","id":"what-the-vendors-and-institutions-now-say","runs":[{"text":"What the vendors and institutions now say"}]},{"t":"p","runs":[{"text":"Several major providers state that their scores are indicative rather than conclusive and should not be the sole basis of an allegation. OpenAI withdrew its own classifier in 2023, citing low accuracy. A number of universities have disabled detector tools in their submission systems for exactly these reasons, and others have adopted policies stating that a detector score alone cannot support a finding."}]},{"t":"p","runs":[{"text":"So if a score is being used against you as though it were proof, that is not only unfair: it is usually contrary to the tool’s own documentation, and saying so calmly is a legitimate part of a response."}]},{"t":"h2","id":"what-this-does-not-mean","runs":[{"text":"What this does not mean"}]},{"t":"p","runs":[{"text":"It does not mean nobody can tell. Markers who know a subject and have read your previous work recognise a change in register, an argument that does not connect to the seminar you were in, a bibliography with fabricated entries, and an answer that cannot be defended in conversation. Those are evidence, and they are how most cases are actually made and how most are correctly made."}]},{"t":"p","runs":[{"text":"It also does not mean generated submissions go undetected in practice. The most reliable methods are the human ones: a viva, a question about your own paragraph, an in-person written task compared against your coursework."}]},{"t":"h2","id":"what-to-do-about-it","runs":[{"text":"What to do about it"}]},{"t":"p","runs":[{"text":"You cannot control whether a detector flags you. You can control whether you have evidence, and that is the whole of the next lesson."}]},{"t":"ul","items":[[{"text":"Write in a tool with version history and leave it on."}],[{"text":"Keep drafts, notes, and the reading you did."}],[{"text":"Keep the unaided-check log from Module 3."}],[{"text":"If your writing is naturally formulaic, or you are writing in a second language, be especially deliberate about this. The system’s error falls on you disproportionately, and the only available protection is a record."}]]},{"t":"p","runs":[{"text":"That is an unjust distribution of effort. It is also, currently, the situation."}]}],"check":{"question":[{"text":"A detector is 95 per cent accurate in both directions and 5 per cent of a 1,000-essay cohort used AI. Why does a flag tell a marker so little?"}],"options":[{"text":[{"text":"Because 95 per cent accuracy is measured on benchmark datasets that do not resemble student coursework."}],"correct":false,"tempting":[{"text":"Raises a real external-validity concern that would make things worse, but the arithmetic already fails at the stated accuracy."}]},{"text":[{"text":"Because a 5 per cent prevalence estimate is itself a guess, and without knowing the true rate no inference can be drawn from a positive result."}],"correct":false,"tempting":[{"text":"Uncertainty in the prevalence widens the range but does not remove the effect, which holds across every plausible value."}]},{"text":[{"text":"Because accuracy figures describe the tool’s performance on average and cannot be applied to an individual essay at all."}],"correct":false,"tempting":[{"text":"Overstates into a claim that no probabilistic evidence is ever informative about a case; the point is the size of the resulting probability, not its inapplicability."}]},{"text":[{"text":"Because the innocent group is so much larger that its false positives roughly equal the true positives."}],"correct":true,"tempting":null}],"order":[2,3,1,0],"why":null},"prev":{"slug":"writing-a-disclosure","title":"Writing a disclosure that helps you","minutes":8},"next":{"slug":"the-process-record","title":"The process record","minutes":8}}}