{"course":{"slug":"ai-actually-explained","title":"AI, Actually Explained","url":"https://addaly.com/learn/ai-actually-explained","level":"beginner","track":"start"},"lesson":{"slug":"deepfakes-and-evidence","title":"Fakes, detectors that do not work, and what is left","url":"https://addaly.com/learn/ai-actually-explained/deepfakes-and-evidence","position":40,"of":92,"minutes":9,"answer":"Detection is unreliable and text detectors misclassify non-native English writing at high rates, so verification has to rest on source, corroboration and provenance rather than on inspecting the file.","keyIdea":"Detection is unreliable and text detectors misclassify non-native English writing at high rates, so verification has to rest on source, corroboration and provenance rather than on inspecting the file."},"body":{"format":"markdown","text":"## The asymmetry\n\nMaking convincing synthetic media has become cheap and easy. Detecting it reliably has not. That gap is the whole problem, and it is unlikely to close, because every detector becomes training material for the next generator.\n\nThree things follow, and they are more useful than any list of tells.\n\n## Detectors are worse than they sound\n\n**For images and video**, detectors report high accuracy on the datasets they were built against and degrade sharply in the wild. Compression, resizing, screenshotting and re-uploading (everything that happens when media travels through a messaging app) strips the artefacts detectors rely on. The traditional visual tells are also gone: hands and text in images were reliable giveaways in 2023 and largely are not now.\n\n**For text, detectors do not work well enough to use on people.** This is the one to be firm about. Studies have found high false-positive rates, and crucially a systematic bias: a 2023 study in *Patterns* ran essays by non-native English writers through seven detectors and found more than half were misclassified as AI-generated, while essays by native speakers were classified correctly far more often. The mechanism is simple: detectors flag low variation in word choice and sentence structure, which is also what second-language writing looks like.\n\n> Figure: Text detectors, run on writing that was entirely human. Bars (% wrongly flagged as AI-generated): Essays by non-native English writers 61; The same essays, reworded in richer vocabulary 12; Essays by native English writers 5. Rounded from the 2023 study in Patterns. Every essay here was written by a person. Detectors flag low variation in word choice and sentence structure, which is also what second-language writing looks like; so an institution deciding honesty cases on a detector's percentage is punishing people for how they write.\n\nOpenAI withdrew its own text classifier in 2023 citing low accuracy. Any institution deciding academic honesty cases on a detector's percentage is punishing people for how they write, disproportionately those writing in a second language. If you are ever accused on this basis, the study above is the citation to have.\n\n## Provenance is a better idea, with real limits\n\nThe more promising direction is not detecting fakes but proving origins. **C2PA Content Credentials** attach a cryptographically signed record to a file describing where it came from and what was done to it. Some cameras, editing tools and generators now support it, and several platforms display it.\n\nThe limits are honest ones. The credential can be stripped by screenshotting, re-encoding or a crop, and most platforms strip metadata on upload. It proves what a signed chain says, not that an image is truthful: a real photograph of a staged scene is authentically a photograph. And its coverage is partial, so absence of a credential means nothing at all. Provenance can eventually raise confidence in marked media. It will not lower confidence in unmarked media, because most media is unmarked.\n\n## The bigger effect: the liar's dividend\n\nThe harm most people expect is being deceived by a fake. The larger and better-documented harm is the reverse: once everyone knows convincing fakes exist, **genuine evidence can be denied.**\n\nA recording of something real is met with \"that is AI\". This has already been argued in court, in politics and in workplace disputes. It costs the denier nothing and it shifts a burden that used to sit on them.\n\nThis matters because it changes what verification is for. You are not primarily defending against clever fakes. You are defending the ability to establish that anything happened.\n\n## What actually works\n\nNone of it is technical, and all of it is old.\n\n**Verify the source, not the file.** Where did this come from? Who first posted it? Does an organisation with something to lose stand behind it? A video from a named news agency with a byline is a different object from the same video forwarded four times.\n\n**Look for independent corroboration.** Two accounts from parties who do not share an interest. A photograph of the same event from another angle. A written record with a date.\n\n**Reverse image search.** Free, thirty seconds, and it catches the single most common case by a wide margin: a real old image recycled with a false caption. Most viral \"fakes\" are not generated at all: they are real media, misdescribed.\n\n**Ask who benefits.** Fabrications are made for reasons, and the reason usually points at someone.\n\n**Slow down on anything that makes you angry.** Emotional arousal is the delivery mechanism, whether the content is generated or not. The pause is the defence.\n\n## The summary\n\nYou cannot reliably tell by looking, and you cannot reliably tell with a tool. What remains is provenance, corroboration and source: the same standards journalism and courts used before any of this existed, now needed for material that used to carry its own credibility."},"check":{"question":"A university uses an AI-text detector to decide plagiarism cases. Beyond general accuracy concerns, what is the specific documented harm?","options":[{"text":"Detectors can be defeated by paraphrasing tools, so determined cheats are not caught while honest students are.","correct":false,"tempting":"Names a real evasion route, but describes ineffectiveness rather than the measured harm to a particular group."},{"text":"Detectors flag low variation in word choice and sentence structure, which is characteristic of second-language writing, so non-native English writers are falsely accused at much higher rates.","correct":true},{"text":"Detectors require uploading student work to a third party, which breaches data protection rules.","correct":false,"tempting":"Raises a genuine privacy concern that is separate from the accuracy question and applies equally to all students."},{"text":"Detectors cannot process handwritten or scanned submissions, so they are inapplicable to most coursework.","correct":false,"tempting":"Invents a format limitation rather than engaging with how the classifier's signal maps onto writing style."}]},"prev":{"slug":"the-model-changed-under-you","title":"The thing you tested is not the thing running today","url":"https://addaly.com/learn/ai-actually-explained/the-model-changed-under-you"},"next":{"slug":"weaker-in-your-language","title":"It is measurably weaker in most languages","url":"https://addaly.com/learn/ai-actually-explained/weaker-in-your-language"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Fakes, detectors that do not work, and what is left', in AI, Actually Explained. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-actually-explained/deepfakes-and-evidence (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Fakes, detectors that do not work, and what is left","courseTitle":"AI, Actually Explained","module":{"index":4,"slug":"why-it-goes-wrong","title":"Why it gets things wrong"},"answer":null,"keyIdea":"Detection is unreliable and text detectors misclassify non-native English writing at high rates, so verification has to rest on source, corroboration and provenance rather than on inspecting the file.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"bars","unit":"% wrongly flagged as AI-generated","title":"Text detectors, run on writing that was entirely human","series":[{"label":"Essays by non-native English writers","value":61,"highlight":true},{"label":"The same essays, reworded in richer vocabulary","value":12},{"label":"Essays by native English writers","value":5}],"caption":"Rounded from the 2023 study in Patterns. Every essay here was written by a person. Detectors flag low variation in word choice and sentence structure, which is also what second-language writing looks like; so an institution deciding honesty cases on a detector’s percentage is punishing people for how they write."},"alt":"Figure: Text detectors, run on writing that was entirely human. Bars (% wrongly flagged as AI-generated): Essays by non-native English writers 61; The same essays, reworded in richer vocabulary 12; Essays by native English writers 5. Rounded from the 2023 study in Patterns. Every essay here was written by a person. Detectors flag low variation in word choice and sentence structure, which is also what second-language writing looks like; so an institution deciding honesty cases on a detector’s percentage is punishing people for how they write."}},"blocks":[{"t":"h2","id":"the-asymmetry","runs":[{"text":"The asymmetry"}]},{"t":"p","runs":[{"text":"Making convincing synthetic media has become cheap and easy. Detecting it reliably has not. That gap is the whole problem, and it is unlikely to close, because every detector becomes training material for the next generator."}]},{"t":"p","runs":[{"text":"Three things follow, and they are more useful than any list of tells."}]},{"t":"h2","id":"detectors-are-worse-than-they-sound","runs":[{"text":"Detectors are worse than they sound"}]},{"t":"p","runs":[{"text":"For images and video","b":true},{"text":", detectors report high accuracy on the datasets they were built against and degrade sharply in the wild. Compression, resizing, screenshotting and re-uploading (everything that happens when media travels through a messaging app) strips the artefacts detectors rely on. The traditional visual tells are also gone: hands and text in images were reliable giveaways in 2023 and largely are not now."}]},{"t":"p","runs":[{"text":"For text, detectors do not work well enough to use on people.","b":true},{"text":" This is the one to be firm about. Studies have found high false-positive rates, and crucially a systematic bias: a 2023 study in "},{"text":"Patterns","i":true},{"text":" ran essays by non-native English writers through seven detectors and found more than half were misclassified as AI-generated, while essays by native speakers were classified correctly far more often. The mechanism is simple: detectors flag low variation in word choice and sentence structure, which is also what second-language writing looks like."}]},{"t":"p","runs":[{"text":"OpenAI withdrew its own text classifier in 2023 citing low accuracy. Any institution deciding academic honesty cases on a detector’s percentage is punishing people for how they write, disproportionately those writing in a second language. If you are ever accused on this basis, the study above is the citation to have."}]},{"t":"h2","id":"provenance-is-a-better-idea-with-real-limits","runs":[{"text":"Provenance is a better idea, with real limits"}]},{"t":"p","runs":[{"text":"The more promising direction is not detecting fakes but proving origins. "},{"text":"C2PA Content Credentials","b":true},{"text":" attach a cryptographically signed record to a file describing where it came from and what was done to it. Some cameras, editing tools and generators now support it, and several platforms display it."}]},{"t":"p","runs":[{"text":"The limits are honest ones. The credential can be stripped by screenshotting, re-encoding or a crop, and most platforms strip metadata on upload. It proves what a signed chain says, not that an image is truthful: a real photograph of a staged scene is authentically a photograph. And its coverage is partial, so absence of a credential means nothing at all. Provenance can eventually raise confidence in marked media. It will not lower confidence in unmarked media, because most media is unmarked."}]},{"t":"h2","id":"the-bigger-effect-the-liar-s-dividend","runs":[{"text":"The bigger effect: the liar’s dividend"}]},{"t":"p","runs":[{"text":"The harm most people expect is being deceived by a fake. The larger and better-documented harm is the reverse: once everyone knows convincing fakes exist, "},{"text":"genuine evidence can be denied.","b":true}]},{"t":"p","runs":[{"text":"A recording of something real is met with “that is AI”. This has already been argued in court, in politics and in workplace disputes. It costs the denier nothing and it shifts a burden that used to sit on them."}]},{"t":"p","runs":[{"text":"This matters because it changes what verification is for. You are not primarily defending against clever fakes. You are defending the ability to establish that anything happened."}]},{"t":"h2","id":"what-actually-works","runs":[{"text":"What actually works"}]},{"t":"p","runs":[{"text":"None of it is technical, and all of it is old."}]},{"t":"p","runs":[{"text":"Verify the source, not the file.","b":true},{"text":" Where did this come from? Who first posted it? Does an organisation with something to lose stand behind it? A video from a named news agency with a byline is a different object from the same video forwarded four times."}]},{"t":"p","runs":[{"text":"Look for independent corroboration.","b":true},{"text":" Two accounts from parties who do not share an interest. A photograph of the same event from another angle. A written record with a date."}]},{"t":"p","runs":[{"text":"Reverse image search.","b":true},{"text":" Free, thirty seconds, and it catches the single most common case by a wide margin: a real old image recycled with a false caption. Most viral “fakes” are not generated at all: they are real media, misdescribed."}]},{"t":"p","runs":[{"text":"Ask who benefits.","b":true},{"text":" Fabrications are made for reasons, and the reason usually points at someone."}]},{"t":"p","runs":[{"text":"Slow down on anything that makes you angry.","b":true},{"text":" Emotional arousal is the delivery mechanism, whether the content is generated or not. The pause is the defence."}]},{"t":"h2","id":"the-summary","runs":[{"text":"The summary"}]},{"t":"p","runs":[{"text":"You cannot reliably tell by looking, and you cannot reliably tell with a tool. What remains is provenance, corroboration and source: the same standards journalism and courts used before any of this existed, now needed for material that used to carry its own credibility."}]}],"check":{"question":[{"text":"A university uses an AI-text detector to decide plagiarism cases. Beyond general accuracy concerns, what is the specific documented harm?"}],"options":[{"text":[{"text":"Detectors can be defeated by paraphrasing tools, so determined cheats are not caught while honest students are."}],"correct":false,"tempting":[{"text":"Names a real evasion route, but describes ineffectiveness rather than the measured harm to a particular group."}]},{"text":[{"text":"Detectors flag low variation in word choice and sentence structure, which is characteristic of second-language writing, so non-native English writers are falsely accused at much higher rates."}],"correct":true,"tempting":null},{"text":[{"text":"Detectors require uploading student work to a third party, which breaches data protection rules."}],"correct":false,"tempting":[{"text":"Raises a genuine privacy concern that is separate from the accuracy question and applies equally to all students."}]},{"text":[{"text":"Detectors cannot process handwritten or scanned submissions, so they are inapplicable to most coursework."}],"correct":false,"tempting":[{"text":"Invents a format limitation rather than engaging with how the classifier’s signal maps onto writing style."}]}],"order":[0,2,1,3],"why":null},"prev":{"slug":"the-model-changed-under-you","title":"The thing you tested is not the thing running today","minutes":8},"next":{"slug":"weaker-in-your-language","title":"It is measurably weaker in most languages","minutes":9}}}