{"course":{"slug":"ai-and-you","title":"AI, Safety and What Goes Wrong","url":"https://addaly.com/learn/ai-and-you","level":"beginner","track":"start"},"lesson":{"slug":"standards-and-audits","title":"What an audit can and cannot see","url":"https://addaly.com/learn/ai-and-you/standards-and-audits","position":69,"of":73,"minutes":7,"answer":"A management-system certificate says an organisation has processes, not that a model is safe. Read an audit's scope and access first, and ask what a failing result would have looked like.","keyIdea":"A management-system certificate says an organisation has processes, not that a model is safe. Read an audit's scope and access first, and ask what a failing result would have looked like."},"body":{"format":"markdown","text":"## The frameworks people actually use\n\nBelow the law sits a layer of voluntary standards, and these are what an organisation practically works from.\n\n**The NIST AI Risk Management Framework**, published in 2023 with a generative AI profile added in 2024, is the most widely adopted. It is organised around four functions: **Govern** (policies, roles, accountability), **Map** (context and what could go wrong), **Measure** (analysis and tracking), **Manage** (prioritisation and response). It is voluntary, free, deliberately non-prescriptive, and useful mainly as a structure that stops teams from forgetting a category.\n\n**ISO/IEC 42001**, published in 2023, is an AI management system standard, and unlike NIST's framework it is **certifiable**: an accredited body audits you and issues a certificate. It concerns whether you have a working management system, not whether any particular model is good.\n\n**ISO/IEC 23894** covers AI risk management guidance, and sits alongside.\n\nA certificate against 42001 says an organisation has processes. It does not say a model is accurate, fair or safe.\n\n## Documentation artefacts worth knowing\n\n**Model cards** (proposed by Mitchell and colleagues in 2019): a short document stating intended use, out-of-scope uses, training data at a high level, evaluation results disaggregated by group, and known limitations. The disaggregation is the important part: overall accuracy hides exactly what the second module taught you to look for.\n\n**Datasheets for datasets** (Gebru and colleagues): the equivalent for data: why it was collected, by whom, from where, with what consent, with what known gaps.\n\n**System cards**: the same idea applied to a deployed product rather than a model, covering the whole pipeline including filters and human review.\n\nThese are useful and they are self-reported. Reading one tells you what the organisation chose to disclose, in a format that makes omissions visible to someone who knows what should be there.\n\n## What an audit actually is\n\n**An audit sees what the auditee provides.** Unless there is a legal power of inspection, the auditor receives selected documentation, selected data and selected access. Most AI audits are conducted with the client's cooperation on the client's material.\n\n**Scope is negotiated.** A favourable audit is often a narrow one. Read the scope statement before the conclusions; it is where the interesting information is.\n\n**Standards for what is being tested barely exist.** Financial audit rests on decades of accounting standards. There is no equivalent settled body for AI, so two auditors can reach different conclusions honestly.\n\n**The incentives run the wrong way.** The auditee pays. This problem is well documented in financial and social auditing and there is no reason to expect this field to escape it.\n\nResearchers call the failure mode **audit washing**: a certificate becomes evidence of responsibility and reduces external scrutiny, without changing the system.\n\nNew York City's Local Law 144 is the most instructive case. It requires an annual independent bias audit of automated employment decision tools with published impact ratios. Early research found low compliance and considerable variation in how audits were conducted and what they covered, which tells you both that mandating audits changes something and that mandating them is not sufficient.\n\n## How to read an audit or a card\n\n1. **What was in scope, and what was excluded?**\n2. **Did the auditor have access to the model, the data, and production logs, or to documents about them?**\n3. **Are results disaggregated by group, or reported as a single figure?**\n4. **Are limitations named specifically, or in general language?**\n5. **Who paid, and does the auditor sell remediation services?**\n6. **What would a failing result have looked like?** If no result would have failed, it was not a test.\n\n## What is worth doing anyway\n\nDespite all of the above, the documentation habit is worth adopting, for a reason that has nothing to do with certificates: writing down intended use, out-of-scope use, evaluation by group and known limitations forces a team to discover what it does not know."},"check":{"question":"A vendor presents an ISO/IEC 42001 certificate as evidence that its hiring model is fair. What does the certificate actually establish?","options":[{"text":"That an accredited body has independently tested the model for disparate impact","correct":false,"tempting":"Confuses certifying a management system with testing one model's outcomes, which it does not do."},{"text":"That the vendor meets the EU AI Act's high-risk obligations, the standard's purpose","correct":false,"tempting":"Standards can support a presumption of conformity in specific ways, but a management certificate is not compliance itself."},{"text":"That it runs an audited AI management system, not that this model is fair","correct":true},{"text":"Nothing at all, since every certificate in this field is simply self-reported","correct":false,"tempting":"Overcorrects. ISO/IEC 42001 involves accredited third-party audit. Its limit is its subject, not its independence."}],"why":"ISO/IEC 42001 certifies how an organisation manages AI. It does not test any one model's selection rates."},"prev":{"slug":"who-governs-this","title":"Who is writing the rules","url":"https://addaly.com/learn/ai-and-you/who-governs-this"},"next":{"slug":"incident-reporting","title":"Learning from what went wrong","url":"https://addaly.com/learn/ai-and-you/incident-reporting"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Source and assessment review continues. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What an audit can and cannot see', in AI, Safety and What Goes Wrong. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/ai-and-you/standards-and-audits (Accessed: 21 November 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What an audit can and cannot see","courseTitle":"AI, Safety and What Goes Wrong","module":{"index":9,"slug":"living-with-it","title":"Living with it: cost, governance and your own practice"},"answer":null,"keyIdea":"A management-system certificate says an organisation has processes, not that a model is safe. Read an audit’s scope and access first, and ask what a failing result would have looked like.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-frameworks-people-actually-use","runs":[{"text":"The frameworks people actually use"}]},{"t":"p","runs":[{"text":"Below the law sits a layer of voluntary standards, and these are what an organisation practically works from."}]},{"t":"p","runs":[{"text":"The NIST AI Risk Management Framework","b":true},{"text":", published in 2023 with a generative AI profile added in 2024, is the most widely adopted. It is organised around four functions: "},{"text":"Govern","b":true},{"text":" (policies, roles, accountability), "},{"text":"Map","b":true},{"text":" (context and what could go wrong), "},{"text":"Measure","b":true},{"text":" (analysis and tracking), "},{"text":"Manage","b":true},{"text":" (prioritisation and response). It is voluntary, free, deliberately non-prescriptive, and useful mainly as a structure that stops teams from forgetting a category."}]},{"t":"p","runs":[{"text":"ISO/IEC 42001","b":true},{"text":", published in 2023, is an AI management system standard, and unlike NIST’s framework it is "},{"text":"certifiable","b":true},{"text":": an accredited body audits you and issues a certificate. It concerns whether you have a working management system, not whether any particular model is good."}]},{"t":"p","runs":[{"text":"ISO/IEC 23894","b":true},{"text":" covers AI risk management guidance, and sits alongside."}]},{"t":"p","runs":[{"text":"A certificate against 42001 says an organisation has processes. It does not say a model is accurate, fair or safe."}]},{"t":"h2","id":"documentation-artefacts-worth-knowing","runs":[{"text":"Documentation artefacts worth knowing"}]},{"t":"p","runs":[{"text":"Model cards","b":true},{"text":" (proposed by Mitchell and colleagues in 2019): a short document stating intended use, out-of-scope uses, training data at a high level, evaluation results disaggregated by group, and known limitations. The disaggregation is the important part: overall accuracy hides exactly what the second module taught you to look for."}]},{"t":"p","runs":[{"text":"Datasheets for datasets","b":true},{"text":" (Gebru and colleagues): the equivalent for data: why it was collected, by whom, from where, with what consent, with what known gaps."}]},{"t":"p","runs":[{"text":"System cards","b":true},{"text":": the same idea applied to a deployed product rather than a model, covering the whole pipeline including filters and human review."}]},{"t":"p","runs":[{"text":"These are useful and they are self-reported. Reading one tells you what the organisation chose to disclose, in a format that makes omissions visible to someone who knows what should be there."}]},{"t":"h2","id":"what-an-audit-actually-is","runs":[{"text":"What an audit actually is"}]},{"t":"p","runs":[{"text":"An audit sees what the auditee provides.","b":true},{"text":" Unless there is a legal power of inspection, the auditor receives selected documentation, selected data and selected access. Most AI audits are conducted with the client’s cooperation on the client’s material."}]},{"t":"p","runs":[{"text":"Scope is negotiated.","b":true},{"text":" A favourable audit is often a narrow one. Read the scope statement before the conclusions; it is where the interesting information is."}]},{"t":"p","runs":[{"text":"Standards for what is being tested barely exist.","b":true},{"text":" Financial audit rests on decades of accounting standards. There is no equivalent settled body for AI, so two auditors can reach different conclusions honestly."}]},{"t":"p","runs":[{"text":"The incentives run the wrong way.","b":true},{"text":" The auditee pays. This problem is well documented in financial and social auditing and there is no reason to expect this field to escape it."}]},{"t":"p","runs":[{"text":"Researchers call the failure mode "},{"text":"audit washing","b":true},{"text":": a certificate becomes evidence of responsibility and reduces external scrutiny, without changing the system."}]},{"t":"p","runs":[{"text":"New York City’s Local Law 144 is the most instructive case. It requires an annual independent bias audit of automated employment decision tools with published impact ratios. Early research found low compliance and considerable variation in how audits were conducted and what they covered, which tells you both that mandating audits changes something and that mandating them is not sufficient."}]},{"t":"h2","id":"how-to-read-an-audit-or-a-card","runs":[{"text":"How to read an audit or a card"}]},{"t":"ol","start":1,"items":[[{"text":"What was in scope, and what was excluded?","b":true}],[{"text":"Did the auditor have access to the model, the data, and production logs, or to documents about them?","b":true}],[{"text":"Are results disaggregated by group, or reported as a single figure?","b":true}],[{"text":"Are limitations named specifically, or in general language?","b":true}],[{"text":"Who paid, and does the auditor sell remediation services?","b":true}],[{"text":"What would a failing result have looked like?","b":true},{"text":" If no result would have failed, it was not a test."}]]},{"t":"h2","id":"what-is-worth-doing-anyway","runs":[{"text":"What is worth doing anyway"}]},{"t":"p","runs":[{"text":"Despite all of the above, the documentation habit is worth adopting, for a reason that has nothing to do with certificates: writing down intended use, out-of-scope use, evaluation by group and known limitations forces a team to discover what it does not know."}]}],"check":{"question":[{"text":"A vendor presents an ISO/IEC 42001 certificate as evidence that its hiring model is fair. What does the certificate actually establish?"}],"options":[{"text":[{"text":"That an accredited body has independently tested the model for disparate impact"}],"correct":false,"tempting":[{"text":"Confuses certifying a management system with testing one model’s outcomes, which it does not do."}]},{"text":[{"text":"That the vendor meets the EU AI Act’s high-risk obligations, the standard’s purpose"}],"correct":false,"tempting":[{"text":"Standards can support a presumption of conformity in specific ways, but a management certificate is not compliance itself."}]},{"text":[{"text":"That it runs an audited AI management system, not that this model is fair"}],"correct":true,"tempting":null},{"text":[{"text":"Nothing at all, since every certificate in this field is simply self-reported"}],"correct":false,"tempting":[{"text":"Overcorrects. ISO/IEC 42001 involves accredited third-party audit. Its limit is its subject, not its independence."}]}],"order":[3,1,0,2],"why":[{"text":"ISO/IEC 42001 certifies how an organisation manages AI. It does not test any one model’s selection rates."}]},"prev":{"slug":"who-governs-this","title":"Who is writing the rules","minutes":10},"next":{"slug":"incident-reporting","title":"Learning from what went wrong","minutes":8}}}