{"course":{"slug":"agents-and-automation","title":"Agents and Automation","url":"https://addaly.com/learn/agents-and-automation","level":"intermediate","track":"build"},"lesson":{"slug":"telling-people-what-is-automated","title":"Saying so, and the tools that cannot prove it","url":"https://addaly.com/learn/agents-and-automation/telling-people-what-is-automated","position":69,"of":72,"minutes":9,"answer":"Text detectors are unreliable and flag non-native writers disproportionately, so disclosure has to be a record you keep rather than a property anyone can test afterwards.","keyIdea":"Text detectors are unreliable and flag non-native writers disproportionately, so disclosure has to be a record you keep rather than a property anyone can test afterwards."},"body":{"format":"markdown","text":"## The cheap version of a hard conversation\n\nPeople mind less about talking to an automated system than teams expect. What they mind is being deceived about it, and being unable to reach a person when the automated answer is wrong.\n\nWhich gives three things to provide, and they are not expensive:\n\n1. **That it is automated.** One line, at the point of contact, in the customer's language.\n2. **How to reach a person.** Not buried, not conditional on failing a series of prompts. A route that works.\n3. **On what basis a decision was made**, if the decision affected them. Not the model's internal workings: the inputs and the rule. \"Your claim was routed for manual review because the amount exceeded the automatic threshold\" is a complete and honest answer.\n\nThe economics are one-sided. An undisclosed automation that is discovered costs an apology, a news cycle in some cases, and a durable loss of trust. Disclosure costs a sentence.\n\n## What not to do\n\nDo not give the automation a human name, a photograph and a first-person voice that implies a person exists. A name and a face land in the reader's mind before any badge does, and by the time they read the small print they have already formed the belief. If it is a system, let it read as a system.\n\nDo not claim a person reviewed something when the review was a click. If a human sign-off is part of your compliance story, the sign-off has to be a real act with real time in it, or the story is false in a way that will not survive an investigation.\n\nDo not present a confidence figure to a customer. It looks like a measurement of correctness and it is not one.\n\n## Internally, tell people before\n\nThe team whose work changes should hear it from you before they see it in their queue. Say what is changing, what is not, what you do not yet know, and when you will next update them.\n\nThe temptation is to promise that nobody's job changes. Do not promise what you have not been told. If the intention is to redeploy people rather than reduce headcount, say so and name who decided it. If nobody has decided yet, say that instead: \"no decision has been made and I will tell you when one is\" is uncomfortable and it is believed, which is more than can be said for reassurance that later turns out to be wrong.\n\nThis is not only ethics. A team that suspects an automation is aimed at them will not tell you about the exceptions, and the exceptions are the project.\n\n## Provenance and watermarking\n\nFor images, audio and video, content-provenance standards such as C2PA attach signed metadata describing how a file was produced. This works when it is preserved, and metadata is routinely stripped by platforms, re-encoding and screenshots. It is a useful chain-of-custody tool inside a system you control and a weak guarantee out in the world.\n\nFor text, be blunt: **there is no reliable way to detect whether text was machine-generated.** Published detectors report accuracy figures that do not hold on ordinary edited writing, and studies have found they flag writing by non-native English speakers at substantially higher rates, because the features they key on (limited vocabulary variation, regular sentence structure) describe careful second-language writing as much as they describe a model.\n\nThe consequences are practical. Never use a detector to accuse anybody of anything, in a school, a workplace or a hiring process. And do not rely on one to police your own outputs.\n\nSo disclosure cannot be something a reader tests. It has to be something you record: a field on every generated artefact saying which system, which config version and which run produced it, kept with the artefact and available on request. That record is the honest substitute for a property nobody can measure from the outside.\n\n## Where the line sits for you\n\nWrite your own rule down, before you need it, and make it short enough to remember. A serviceable one:\n\n> Anything a person receives that appears to come from us says whether a machine wrote it. Any decision affecting a person can be appealed to a human being, and we can say what the decision was based on. We do not pretend a system is a person, ever.\n\nThree sentences. They will settle more arguments than a policy document, because they can be quoted from memory by somebody making a decision at speed, which is when these questions actually arise."},"check":{"question":"A university wants to check whether submitted essays were machine-written, and a vendor offers a detector claiming 98 per cent accuracy. What is the strongest reason not to use it for accusations?","options":[{"text":"Detectors can be defeated by paraphrasing tools, so determined students would evade it anyway.","correct":false,"tempting":"True and important for effectiveness, but it explains why the tool fails at its job rather than why using it causes harm."},{"text":"Reported accuracy does not hold on ordinary edited writing, and detectors flag non-native English writers disproportionately, so the errors fall on identifiable people.","correct":true},{"text":"The vendor’s figure was measured on their own benchmark, so an independent evaluation is needed first.","correct":false,"tempting":"A reasonable procedural objection, but it implies the problem is verification when independent evaluation has repeatedly found the same failures."},{"text":"Using a detector processes student work through a third party, which raises data-protection questions.","correct":false,"tempting":"A genuine concern worth resolving separately; it does not address whether the output is trustworthy enough to act on."}]},"prev":{"slug":"when-people-route-around-it","title":"When people quietly work around it","url":"https://addaly.com/learn/agents-and-automation/when-people-route-around-it"},"next":{"slug":"rules-that-differ-by-country","title":"The rules, where they agree and where they do not","url":"https://addaly.com/learn/agents-and-automation/rules-that-differ-by-country"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Saying so, and the tools that cannot prove it', in Agents and Automation. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/agents-and-automation/telling-people-what-is-automated (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Saying so, and the tools that cannot prove it","courseTitle":"Agents and Automation","module":{"index":8,"slug":"living-with-it","title":"The year after it ships"},"answer":null,"keyIdea":"Text detectors are unreliable and flag non-native writers disproportionately, so disclosure has to be a record you keep rather than a property anyone can test afterwards.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-cheap-version-of-a-hard-conversation","runs":[{"text":"The cheap version of a hard conversation"}]},{"t":"p","runs":[{"text":"People mind less about talking to an automated system than teams expect. What they mind is being deceived about it, and being unable to reach a person when the automated answer is wrong."}]},{"t":"p","runs":[{"text":"Which gives three things to provide, and they are not expensive:"}]},{"t":"ol","start":1,"items":[[{"text":"That it is automated.","b":true},{"text":" One line, at the point of contact, in the customer’s language."}],[{"text":"How to reach a person.","b":true},{"text":" Not buried, not conditional on failing a series of prompts. A route that works."}],[{"text":"On what basis a decision was made","b":true},{"text":", if the decision affected them. Not the model’s internal workings: the inputs and the rule. “Your claim was routed for manual review because the amount exceeded the automatic threshold” is a complete and honest answer."}]]},{"t":"p","runs":[{"text":"The economics are one-sided. An undisclosed automation that is discovered costs an apology, a news cycle in some cases, and a durable loss of trust. Disclosure costs a sentence."}]},{"t":"h2","id":"what-not-to-do","runs":[{"text":"What not to do"}]},{"t":"p","runs":[{"text":"Do not give the automation a human name, a photograph and a first-person voice that implies a person exists. A name and a face land in the reader’s mind before any badge does, and by the time they read the small print they have already formed the belief. If it is a system, let it read as a system."}]},{"t":"p","runs":[{"text":"Do not claim a person reviewed something when the review was a click. If a human sign-off is part of your compliance story, the sign-off has to be a real act with real time in it, or the story is false in a way that will not survive an investigation."}]},{"t":"p","runs":[{"text":"Do not present a confidence figure to a customer. It looks like a measurement of correctness and it is not one."}]},{"t":"h2","id":"internally-tell-people-before","runs":[{"text":"Internally, tell people before"}]},{"t":"p","runs":[{"text":"The team whose work changes should hear it from you before they see it in their queue. Say what is changing, what is not, what you do not yet know, and when you will next update them."}]},{"t":"p","runs":[{"text":"The temptation is to promise that nobody’s job changes. Do not promise what you have not been told. If the intention is to redeploy people rather than reduce headcount, say so and name who decided it. If nobody has decided yet, say that instead: “no decision has been made and I will tell you when one is” is uncomfortable and it is believed, which is more than can be said for reassurance that later turns out to be wrong."}]},{"t":"p","runs":[{"text":"This is not only ethics. A team that suspects an automation is aimed at them will not tell you about the exceptions, and the exceptions are the project."}]},{"t":"h2","id":"provenance-and-watermarking","runs":[{"text":"Provenance and watermarking"}]},{"t":"p","runs":[{"text":"For images, audio and video, content-provenance standards such as C2PA attach signed metadata describing how a file was produced. This works when it is preserved, and metadata is routinely stripped by platforms, re-encoding and screenshots. It is a useful chain-of-custody tool inside a system you control and a weak guarantee out in the world."}]},{"t":"p","runs":[{"text":"For text, be blunt: "},{"text":"there is no reliable way to detect whether text was machine-generated.","b":true},{"text":" Published detectors report accuracy figures that do not hold on ordinary edited writing, and studies have found they flag writing by non-native English speakers at substantially higher rates, because the features they key on (limited vocabulary variation, regular sentence structure) describe careful second-language writing as much as they describe a model."}]},{"t":"p","runs":[{"text":"The consequences are practical. Never use a detector to accuse anybody of anything, in a school, a workplace or a hiring process. And do not rely on one to police your own outputs."}]},{"t":"p","runs":[{"text":"So disclosure cannot be something a reader tests. It has to be something you record: a field on every generated artefact saying which system, which config version and which run produced it, kept with the artefact and available on request. That record is the honest substitute for a property nobody can measure from the outside."}]},{"t":"h2","id":"where-the-line-sits-for-you","runs":[{"text":"Where the line sits for you"}]},{"t":"p","runs":[{"text":"Write your own rule down, before you need it, and make it short enough to remember. A serviceable one:"}]},{"t":"quote","runs":[{"text":"Anything a person receives that appears to come from us says whether a machine wrote it. Any decision affecting a person can be appealed to a human being, and we can say what the decision was based on. We do not pretend a system is a person, ever."}]},{"t":"p","runs":[{"text":"Three sentences. They will settle more arguments than a policy document, because they can be quoted from memory by somebody making a decision at speed, which is when these questions actually arise."}]}],"check":{"question":[{"text":"A university wants to check whether submitted essays were machine-written, and a vendor offers a detector claiming 98 per cent accuracy. What is the strongest reason not to use it for accusations?"}],"options":[{"text":[{"text":"Detectors can be defeated by paraphrasing tools, so determined students would evade it anyway."}],"correct":false,"tempting":[{"text":"True and important for effectiveness, but it explains why the tool fails at its job rather than why using it causes harm."}]},{"text":[{"text":"Reported accuracy does not hold on ordinary edited writing, and detectors flag non-native English writers disproportionately, so the errors fall on identifiable people."}],"correct":true,"tempting":null},{"text":[{"text":"The vendor’s figure was measured on their own benchmark, so an independent evaluation is needed first."}],"correct":false,"tempting":[{"text":"A reasonable procedural objection, but it implies the problem is verification when independent evaluation has repeatedly found the same failures."}]},{"text":[{"text":"Using a detector processes student work through a third party, which raises data-protection questions."}],"correct":false,"tempting":[{"text":"A genuine concern worth resolving separately; it does not address whether the output is trustworthy enough to act on."}]}],"order":[2,1,0,3],"why":null},"prev":{"slug":"when-people-route-around-it","title":"When people quietly work around it","minutes":9},"next":{"slug":"rules-that-differ-by-country","title":"The rules, where they agree and where they do not","minutes":10}}}