{"course":{"slug":"building-apps-with-ai","title":"Building Apps with AI","url":"https://addaly.com/learn/building-apps-with-ai","level":"beginner","track":"build"},"lesson":{"slug":"asking-the-model-to-audit-itself","title":"Asking the model to audit itself, and what it cannot see","url":"https://addaly.com/learn/building-apps-with-ai/asking-the-model-to-audit-itself","position":77,"of":87,"minutes":8,"answer":"A model reviewing code finds the holes that are visible in the code (a missing check, a raw query.","keyIdea":"A model reviewing code finds the holes that are visible in the code (a missing check, a raw query, a spread body) and cannot find the ones that live in what the code does not say, so the review is a per-route checklist plus a free scanner plus a person when the stakes are high."},"body":{"format":"markdown","text":"## The general review\n\n\"Check this app for security problems\" returns a paragraph. It mentions input validation, recommends HTTPS, suggests keeping dependencies updated, and concludes that no critical issues were found. It is not wrong; it is not a review. A model given a whole codebase and a vague question produces the summary of security advice it has read most often, and the summary is the same for every codebase.\n\nThe first security lesson ended with a specific question: for every route, which line checks ownership. This lesson widens that into the review the model can actually do, and marks the edge of what it can do at all.\n\n## The checklist, per route\n\nFor each file that handles a request, five questions, asked as one prompt with the file in context:\n\n```text\nFor this route, answer each with a line number or \"none\":\n1. Which line establishes who is making the request, from the session?\n2. Which line decides whether that person may perform this action on this record?\n3. Which line validates the body or parameters against a schema, and does anything\n   below it use a raw body or raw parameter?\n4. Which lines build SQL, a shell command, or a URL the server fetches, and is any\n   value in them derived from the request?\n5. What does this route return when the check in 2 fails, and what does it log?\n```\n\nThe answers are checkable. A \"none\" on question 2 for a route that touches a user's record is a finding. A line number on question 3 that sits below a database call is a finding. This is the review the model is good at: reading code it can see and locating the presence or absence of a pattern. Run it over every route, and keep the output, because next month's review is a diff against it.\n\n## What no review of the code can see\n\nThree categories are invisible in the text, whoever reads it.\n\n**What the code does not say.** The list route in this lesson's question is ordinary working code. Nothing in it is wrong as code; what is wrong is what it omits, and an omission is visible only to a reader who knows what should be there. The checklist supplies that knowledge for the common cases. For the uncommon (the workflow where a refund can be requested twice, the invite that can be accepted by the wrong person) you have to describe what should be true and ask whether it is enforced, and that description is the specification block's job.\n\n**What lives elsewhere.** The host's settings, the bucket's public flag, the database role's privileges, the DNS, the provider's retention terms. None of it is in the repository. A code review cannot tell you the bucket is public.\n\n**What is true at runtime.** A dependency with a vulnerability disclosed yesterday; a header the framework sets that the CDN strips; the middleware that a crafted request skips. Reading code says what it intends. Only running it says what it does.\n\n## Free tools for the rest\n\n`npm audit` for known vulnerabilities in dependencies, read with the honesty the dependency lesson asked for.\n\n`gitleaks` for secrets in the history.\n\n`semgrep` (free for local use) runs pattern rules over the code and finds string-built SQL, raw HTML injection, and a few hundred other shapes, faster and more consistently than a model reading file by file.\n\nOWASP ZAP (free, desktop or command line) is a scanner that attacks a running site: a baseline scan against the preview deployment takes ten minutes, needs no configuration, and reports missing headers, exposed files and reflected input. Point it at the preview, never at production, and never at a site that is not yours.\n\nA `security.txt` file at `/.well-known/security.txt` with an email address, so that a stranger who finds something has a way to tell you that is not a public post. Costs nothing, and it is the difference between a quiet email and a screenshot on social media.\n\n## When to pay a person\n\nOther people's money, health data, anything involving children, anything with a legal duty: the categories the closing lesson of this course names as unfit for unsupervised AI code. For these, a penetration test by a person is not optional, and it is not what a model or a scanner does: a tester chains small findings into a real path, reasons about your business rather than about patterns, and tells you what an attacker would actually do first. It costs real money, from a few hundred dollars for a small scoped test upward, and it is cheaper than the alternative in every case where the alternative has happened.\n\n## The review as a habit\n\nBefore each release: the per-route checklist on changed routes, the three-account script, the bundle and history scan, `npm audit`, and ten minutes of ZAP against the preview. An hour, most of it automated. The generated app started with the holes the first lesson of this block listed; this is the routine that keeps it from acquiring them again, one feature at a time."},"check":{"question":"You ask an agent to review the app for security and it reports 'no critical issues found'. The app has a list route that returns every user's bookings with the filtering done on the page. Why did the review miss it?","options":[{"text":"The agent only reviews files it has recently edited or that were open in the editor during the session, and that route was written weeks ago in a folder it had no reason to open, so it was never in the context at all.","correct":false,"tempting":"Reviews read whatever is in the context; the age of the file is not the mechanism, though a route outside the context is missed for that reason."},{"text":"Models are not trained on security material and cannot recognise this class of hole.","correct":false,"tempting":"They recognise it readily when asked the specific question; the miss is in the framing of the request, not the model's knowledge."},{"text":"A general request produces a general answer; nothing in the prompt asked, route by route, which line restricts the rows to the requesting user, and the route looks like ordinary working code.","correct":true},{"text":"The filtering on the page is a valid design, so the review was correct to pass it.","correct":false,"tempting":"Filtering in the browser after sending every row means every row was sent; the Network tab shows the whole table."}]},"prev":{"slug":"the-admin-page-nobody-protected","title":"The admin page nobody protected","url":"https://addaly.com/learn/building-apps-with-ai/the-admin-page-nobody-protected"},"next":{"slug":"what-it-costs","title":"What it actually costs","url":"https://addaly.com/learn/building-apps-with-ai/what-it-costs"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Asking the model to audit itself, and what it cannot see', in Building Apps with AI. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/building-apps-with-ai/asking-the-model-to-audit-itself (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Asking the model to audit itself, and what it cannot see","courseTitle":"Building Apps with AI","module":{"index":8,"slug":"the-holes-ai-code-has-by-default","title":"The security holes AI code has by default"},"answer":null,"keyIdea":"A model reviewing code finds the holes that are visible in the code (a missing check, a raw query, a spread body) and cannot find the ones that live in what the code does not say, so the review is a per-route checklist plus a free scanner plus a person when the stakes are high.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-general-review","runs":[{"text":"The general review"}]},{"t":"p","runs":[{"text":"“Check this app for security problems” returns a paragraph. It mentions input validation, recommends HTTPS, suggests keeping dependencies updated, and concludes that no critical issues were found. It is not wrong; it is not a review. A model given a whole codebase and a vague question produces the summary of security advice it has read most often, and the summary is the same for every codebase."}]},{"t":"p","runs":[{"text":"The first security lesson ended with a specific question: for every route, which line checks ownership. This lesson widens that into the review the model can actually do, and marks the edge of what it can do at all."}]},{"t":"h2","id":"the-checklist-per-route","runs":[{"text":"The checklist, per route"}]},{"t":"p","runs":[{"text":"For each file that handles a request, five questions, asked as one prompt with the file in context:"}]},{"t":"code","lang":"text","label":"Text","text":"For this route, answer each with a line number or \"none\":\n1. Which line establishes who is making the request, from the session?\n2. Which line decides whether that person may perform this action on this record?\n3. Which line validates the body or parameters against a schema, and does anything\n   below it use a raw body or raw parameter?\n4. Which lines build SQL, a shell command, or a URL the server fetches, and is any\n   value in them derived from the request?\n5. What does this route return when the check in 2 fails, and what does it log?"},{"t":"p","runs":[{"text":"The answers are checkable. A “none” on question 2 for a route that touches a user’s record is a finding. A line number on question 3 that sits below a database call is a finding. This is the review the model is good at: reading code it can see and locating the presence or absence of a pattern. Run it over every route, and keep the output, because next month’s review is a diff against it."}]},{"t":"h2","id":"what-no-review-of-the-code-can-see","runs":[{"text":"What no review of the code can see"}]},{"t":"p","runs":[{"text":"Three categories are invisible in the text, whoever reads it."}]},{"t":"p","runs":[{"text":"What the code does not say.","b":true},{"text":" The list route in this lesson’s question is ordinary working code. Nothing in it is wrong as code; what is wrong is what it omits, and an omission is visible only to a reader who knows what should be there. The checklist supplies that knowledge for the common cases. For the uncommon (the workflow where a refund can be requested twice, the invite that can be accepted by the wrong person) you have to describe what should be true and ask whether it is enforced, and that description is the specification block’s job."}]},{"t":"p","runs":[{"text":"What lives elsewhere.","b":true},{"text":" The host’s settings, the bucket’s public flag, the database role’s privileges, the DNS, the provider’s retention terms. None of it is in the repository. A code review cannot tell you the bucket is public."}]},{"t":"p","runs":[{"text":"What is true at runtime.","b":true},{"text":" A dependency with a vulnerability disclosed yesterday; a header the framework sets that the CDN strips; the middleware that a crafted request skips. Reading code says what it intends. Only running it says what it does."}]},{"t":"h2","id":"free-tools-for-the-rest","runs":[{"text":"Free tools for the rest"}]},{"t":"p","runs":[{"text":"npm audit","code":true},{"text":" for known vulnerabilities in dependencies, read with the honesty the dependency lesson asked for."}]},{"t":"p","runs":[{"text":"gitleaks","code":true},{"text":" for secrets in the history."}]},{"t":"p","runs":[{"text":"semgrep","code":true},{"text":" (free for local use) runs pattern rules over the code and finds string-built SQL, raw HTML injection, and a few hundred other shapes, faster and more consistently than a model reading file by file."}]},{"t":"p","runs":[{"text":"OWASP ZAP (free, desktop or command line) is a scanner that attacks a running site: a baseline scan against the preview deployment takes ten minutes, needs no configuration, and reports missing headers, exposed files and reflected input. Point it at the preview, never at production, and never at a site that is not yours."}]},{"t":"p","runs":[{"text":"A "},{"text":"security.txt","code":true},{"text":" file at "},{"text":"/.well-known/security.txt","code":true},{"text":" with an email address, so that a stranger who finds something has a way to tell you that is not a public post. Costs nothing, and it is the difference between a quiet email and a screenshot on social media."}]},{"t":"h2","id":"when-to-pay-a-person","runs":[{"text":"When to pay a person"}]},{"t":"p","runs":[{"text":"Other people’s money, health data, anything involving children, anything with a legal duty: the categories the closing lesson of this course names as unfit for unsupervised AI code. For these, a penetration test by a person is not optional, and it is not what a model or a scanner does: a tester chains small findings into a real path, reasons about your business rather than about patterns, and tells you what an attacker would actually do first. It costs real money, from a few hundred dollars for a small scoped test upward, and it is cheaper than the alternative in every case where the alternative has happened."}]},{"t":"h2","id":"the-review-as-a-habit","runs":[{"text":"The review as a habit"}]},{"t":"p","runs":[{"text":"Before each release: the per-route checklist on changed routes, the three-account script, the bundle and history scan, "},{"text":"npm audit","code":true},{"text":", and ten minutes of ZAP against the preview. An hour, most of it automated. The generated app started with the holes the first lesson of this block listed; this is the routine that keeps it from acquiring them again, one feature at a time."}]}],"check":{"question":[{"text":"You ask an agent to review the app for security and it reports ‘no critical issues found’. The app has a list route that returns every user’s bookings with the filtering done on the page. Why did the review miss it?"}],"options":[{"text":[{"text":"The agent only reviews files it has recently edited or that were open in the editor during the session, and that route was written weeks ago in a folder it had no reason to open, so it was never in the context at all."}],"correct":false,"tempting":[{"text":"Reviews read whatever is in the context; the age of the file is not the mechanism, though a route outside the context is missed for that reason."}]},{"text":[{"text":"Models are not trained on security material and cannot recognise this class of hole."}],"correct":false,"tempting":[{"text":"They recognise it readily when asked the specific question; the miss is in the framing of the request, not the model’s knowledge."}]},{"text":[{"text":"A general request produces a general answer; nothing in the prompt asked, route by route, which line restricts the rows to the requesting user, and the route looks like ordinary working code."}],"correct":true,"tempting":null},{"text":[{"text":"The filtering on the page is a valid design, so the review was correct to pass it."}],"correct":false,"tempting":[{"text":"Filtering in the browser after sending every row means every row was sent; the Network tab shows the whole table."}]}],"order":[0,3,1,2],"why":null},"prev":{"slug":"the-admin-page-nobody-protected","title":"The admin page nobody protected","minutes":8},"next":{"slug":"what-it-costs","title":"What it actually costs","minutes":7}}}