{"course":{"slug":"open-models","title":"The Open Model Ecosystem","url":"https://addaly.com/learn/open-models","level":"intermediate","track":"deep"},"lesson":{"slug":"guardrails-as-models","title":"Guardrails as a separate layer","url":"https://addaly.com/learn/open-models/guardrails-as-models","position":49,"of":71,"minutes":8,"answer":"A separate classifier is a different failure surface from the generating model because it labels rather than obeys, so layer cheap deterministic checks, a small input classifier, the model and an output check.","keyIdea":"A separate classifier is a different failure surface from the generating model because it labels rather than obeys, so layer cheap deterministic checks, a small input classifier, the model and an output check, and never let any of it stand between a model and an irreversible action."},"body":{"format":"markdown","text":"## Why the check must not be the same model\n\nA model asked to both answer the user and police itself has a structural problem: the thing it is policing is in its own context, written by the person it is policing. Prompt injection works precisely because instructions and data arrive through the same channel.\n\nA separate classifier does not have that problem to the same degree. It is not following instructions; it is producing a label. Text saying *ignore your previous instructions* is an input to be classified, not a command, because the model was tuned to classify rather than to comply.\n\nThat is not perfect either (classifiers can be evaded), but it is a different failure surface, and layering the two is much stronger than either alone.\n\n## The open guardrail models\n\n**Llama Guard** classifies against a hazard taxonomy (violent crime, sexual content, self-harm, privacy, intellectual property and others) and runs on both the user's input and the model's output. You can edit the taxonomy in the prompt, which is more useful than it sounds: the categories you care about are rarely exactly the default list.\n\n**ShieldGemma** does the same job in the Gemma family, in several sizes including very small ones.\n\n**Granite Guardian** from IBM covers harms plus groundedness and relevance checks for retrieval systems.\n\n**Prompt-injection classifiers** are typically small encoder models (DeBERTa-sized, tens of millions of parameters) trained to spot injection attempts. They run in single-digit milliseconds on a CPU, which is what makes them practical on every request.\n\n**NeMo Guardrails** is orchestration rather than a model: a framework for defining conversational rails and wiring checks around a model.\n\n## Order them by cost\n\nThe pipeline that works is cheapest-first, so the expensive checks only see what survives.\n\n1. **Deterministic checks.** Length limits, allow-lists, regular expressions for obvious patterns, rate limits per user. Microseconds, and they catch a surprising amount.\n2. **A small classifier on the input.** Injection detection and hazard classification. Single-digit milliseconds on a CPU.\n3. **The model.**\n4. **A check on the output.** Hazard classification again, plus whatever your domain requires: a groundedness check for retrieval, a personal-data detector, a schema validation.\n\nStep 4 matters more than people expect. Plenty of inputs look benign and produce output you cannot ship. Checking only the input is the more common design and the weaker one.\n\n## The cost, and how to pay less of it\n\nA guardrail model adds latency to every request. A small classifier at 20 milliseconds against a 2-second generation is 1% and invisible. A large guardrail model doing a full generation to produce a verdict can add hundreds of milliseconds twice, which is a product decision rather than a technical one.\n\nTwo mitigations. **Run the output check concurrently with streaming** and be prepared to withdraw the response: this is what most production systems do. And **use the smallest classifier that hits your target**, measured on your own inputs rather than on the published numbers.\n\n## False positives are the expensive failure\n\nA guardrail that blocks legitimate requests fails your users silently. They do not report it; they leave. Measure both directions, on your own data:\n\n- **False negatives:** attempts that got through. Collect them from red-teaming and from incidents.\n- **False positives:** the false-refusal set from the previous lesson, run through the guardrail rather than the model.\n\nReport both numbers together and tune the threshold to a ratio you chose deliberately, not to whatever the default was.\n\n## What no guardrail gives you\n\nRobustness against a determined adversary. Every published classifier has been evaded, and evasion techniques circulate quickly. Layered defence lowers the rate of successful attacks; it does not make the risk zero.\n\nThe design consequence is architectural rather than model-shaped: **do not put a language model where a successful attack causes irreversible harm.** No amount of classification makes it safe to let a model, on its own, send money, delete data, or publish to the world. Keep a person or a deterministic rule in the path of any action you cannot undo."},"check":{"question":"A team adds a guardrail model that checks user input only, and later finds harmful content in responses even though every input was benign. What does this show?","options":[{"text":"The guardrail model is too small and needs replacing with a larger one.","correct":false,"tempting":"Size affects accuracy on the inputs it sees; it cannot help with a stage the guardrail never inspects."},{"text":"The input classifier was evaded by an injection technique it had not seen.","correct":false,"tempting":"A real category of failure, but the premise states the inputs were genuinely benign, so nothing needed evading."},{"text":"Benign inputs can produce unshippable output, so the output side needs its own check regardless of how good the input classifier is.","correct":true},{"text":"The generating model's own safety tuning has degraded, probably through quantisation.","correct":false,"tempting":"Quantisation can shift refusal behaviour slightly, but the structural gap here is that nothing examined the output at all."}]},"prev":{"slug":"refusals-and-uncensored-models","title":"Refusals, and the uncensored question","url":"https://addaly.com/learn/open-models/refusals-and-uncensored-models"},"next":{"slug":"when-the-model-does-not-know","title":"When the model does not know","url":"https://addaly.com/learn/open-models/when-the-model-does-not-know"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Guardrails as a separate layer', in The Open Model Ecosystem. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/open-models/guardrails-as-models (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Guardrails as a separate layer","courseTitle":"The Open Model Ecosystem","module":{"index":5,"slug":"getting-good-output","title":"Getting good output"},"answer":null,"keyIdea":"A separate classifier is a different failure surface from the generating model because it labels rather than obeys, so layer cheap deterministic checks, a small input classifier, the model and an output check, and never let any of it stand between a model and an irreversible action.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"why-the-check-must-not-be-the-same-model","runs":[{"text":"Why the check must not be the same model"}]},{"t":"p","runs":[{"text":"A model asked to both answer the user and police itself has a structural problem: the thing it is policing is in its own context, written by the person it is policing. Prompt injection works precisely because instructions and data arrive through the same channel."}]},{"t":"p","runs":[{"text":"A separate classifier does not have that problem to the same degree. It is not following instructions; it is producing a label. Text saying "},{"text":"ignore your previous instructions","i":true},{"text":" is an input to be classified, not a command, because the model was tuned to classify rather than to comply."}]},{"t":"p","runs":[{"text":"That is not perfect either (classifiers can be evaded), but it is a different failure surface, and layering the two is much stronger than either alone."}]},{"t":"h2","id":"the-open-guardrail-models","runs":[{"text":"The open guardrail models"}]},{"t":"p","runs":[{"text":"Llama Guard","b":true},{"text":" classifies against a hazard taxonomy (violent crime, sexual content, self-harm, privacy, intellectual property and others) and runs on both the user’s input and the model’s output. You can edit the taxonomy in the prompt, which is more useful than it sounds: the categories you care about are rarely exactly the default list."}]},{"t":"p","runs":[{"text":"ShieldGemma","b":true},{"text":" does the same job in the Gemma family, in several sizes including very small ones."}]},{"t":"p","runs":[{"text":"Granite Guardian","b":true},{"text":" from IBM covers harms plus groundedness and relevance checks for retrieval systems."}]},{"t":"p","runs":[{"text":"Prompt-injection classifiers","b":true},{"text":" are typically small encoder models (DeBERTa-sized, tens of millions of parameters) trained to spot injection attempts. They run in single-digit milliseconds on a CPU, which is what makes them practical on every request."}]},{"t":"p","runs":[{"text":"NeMo Guardrails","b":true},{"text":" is orchestration rather than a model: a framework for defining conversational rails and wiring checks around a model."}]},{"t":"h2","id":"order-them-by-cost","runs":[{"text":"Order them by cost"}]},{"t":"p","runs":[{"text":"The pipeline that works is cheapest-first, so the expensive checks only see what survives."}]},{"t":"ol","start":1,"items":[[{"text":"Deterministic checks.","b":true},{"text":" Length limits, allow-lists, regular expressions for obvious patterns, rate limits per user. Microseconds, and they catch a surprising amount."}],[{"text":"A small classifier on the input.","b":true},{"text":" Injection detection and hazard classification. Single-digit milliseconds on a CPU."}],[{"text":"The model.","b":true}],[{"text":"A check on the output.","b":true},{"text":" Hazard classification again, plus whatever your domain requires: a groundedness check for retrieval, a personal-data detector, a schema validation."}]]},{"t":"p","runs":[{"text":"Step 4 matters more than people expect. Plenty of inputs look benign and produce output you cannot ship. Checking only the input is the more common design and the weaker one."}]},{"t":"h2","id":"the-cost-and-how-to-pay-less-of-it","runs":[{"text":"The cost, and how to pay less of it"}]},{"t":"p","runs":[{"text":"A guardrail model adds latency to every request. A small classifier at 20 milliseconds against a 2-second generation is 1% and invisible. A large guardrail model doing a full generation to produce a verdict can add hundreds of milliseconds twice, which is a product decision rather than a technical one."}]},{"t":"p","runs":[{"text":"Two mitigations. "},{"text":"Run the output check concurrently with streaming","b":true},{"text":" and be prepared to withdraw the response: this is what most production systems do. And "},{"text":"use the smallest classifier that hits your target","b":true},{"text":", measured on your own inputs rather than on the published numbers."}]},{"t":"h2","id":"false-positives-are-the-expensive-failure","runs":[{"text":"False positives are the expensive failure"}]},{"t":"p","runs":[{"text":"A guardrail that blocks legitimate requests fails your users silently. They do not report it; they leave. Measure both directions, on your own data:"}]},{"t":"ul","items":[[{"text":"False negatives:","b":true},{"text":" attempts that got through. Collect them from red-teaming and from incidents."}],[{"text":"False positives:","b":true},{"text":" the false-refusal set from the previous lesson, run through the guardrail rather than the model."}]]},{"t":"p","runs":[{"text":"Report both numbers together and tune the threshold to a ratio you chose deliberately, not to whatever the default was."}]},{"t":"h2","id":"what-no-guardrail-gives-you","runs":[{"text":"What no guardrail gives you"}]},{"t":"p","runs":[{"text":"Robustness against a determined adversary. Every published classifier has been evaded, and evasion techniques circulate quickly. Layered defence lowers the rate of successful attacks; it does not make the risk zero."}]},{"t":"p","runs":[{"text":"The design consequence is architectural rather than model-shaped: "},{"text":"do not put a language model where a successful attack causes irreversible harm.","b":true},{"text":" No amount of classification makes it safe to let a model, on its own, send money, delete data, or publish to the world. Keep a person or a deterministic rule in the path of any action you cannot undo."}]}],"check":{"question":[{"text":"A team adds a guardrail model that checks user input only, and later finds harmful content in responses even though every input was benign. What does this show?"}],"options":[{"text":[{"text":"The guardrail model is too small and needs replacing with a larger one."}],"correct":false,"tempting":[{"text":"Size affects accuracy on the inputs it sees; it cannot help with a stage the guardrail never inspects."}]},{"text":[{"text":"The input classifier was evaded by an injection technique it had not seen."}],"correct":false,"tempting":[{"text":"A real category of failure, but the premise states the inputs were genuinely benign, so nothing needed evading."}]},{"text":[{"text":"Benign inputs can produce unshippable output, so the output side needs its own check regardless of how good the input classifier is."}],"correct":true,"tempting":null},{"text":[{"text":"The generating model’s own safety tuning has degraded, probably through quantisation."}],"correct":false,"tempting":[{"text":"Quantisation can shift refusal behaviour slightly, but the structural gap here is that nothing examined the output at all."}]}],"order":[1,0,2,3],"why":null},"prev":{"slug":"refusals-and-uncensored-models","title":"Refusals, and the uncensored question","minutes":9},"next":{"slug":"when-the-model-does-not-know","title":"When the model does not know","minutes":9}}}