{"course":{"slug":"how-software-changed","title":"How Software Development Changed","url":"https://addaly.com/learn/how-software-changed","level":"intermediate","track":"work"},"lesson":{"slug":"acceptance-criteria-as-tests","title":"Criteria that become test names","url":"https://addaly.com/learn/how-software-changed/acceptance-criteria-as-tests","position":32,"of":87,"minutes":9,"answer":"Replace every adjective in a requirement with an observation somebody could make, and write criteria as worked examples with real values.","keyIdea":"Replace every adjective in a requirement with an observation somebody could make, and write criteria as worked examples with real values, because a specification's expected values are the only ones that come from outside the code."},"body":{"format":"markdown","text":"## Adjectives are not criteria\n\n\"The export should be fast.\" \"The form should be robust.\" \"Handle errors gracefully.\" Each of these passes review in a ticket and specifies nothing, because every reader supplies a different meaning and nobody finds out until the disagreement is expensive.\n\nThe repair is mechanical. Replace each adjective with an observation somebody could make.\n\n- *Fast* → 95th percentile under 300 ms at 50 requests per second, on the production dataset size.\n- *Robust* → a malformed row does not abort the import; it is skipped, counted, and reported in the summary.\n- *Graceful* → on a provider timeout, return 503 with `Retry-After: 30`, log at warning, do not retry a non-idempotent call.\n- *Secure* → a user cannot read another tenant's records through any endpoint in this module, tested with a second tenant's token.\n\nEach rewrite took twenty seconds and each is now something a test can assert and a reviewer can check.\n\n## The Given / When / Then shape\n\nOne format, used consistently, does most of the work. It forces you to state the starting state, which is where the omissions live.\n\n```\nGiven a cart with 3 items totalling ₹2,400\nAnd a promo code SAVE10 valid until 30 September\nWhen the customer applies SAVE10 on 29 September\nThen the total becomes ₹2,160\nAnd the discount line reads \"SAVE10 −₹240\"\n```\n\nThen the negative cases, which is where the real specification lives:\n\n```\nGiven the same cart\nWhen the customer applies SAVE10 on 1 October\nThen the code is rejected with \"This code has expired\"\nAnd the total is unchanged\n\nGiven a cart with SAVE10 already applied\nWhen the customer applies SAVE10 again\nThen the discount is not applied twice\n```\n\nThree criteria and you have already decided expiry semantics, idempotence and the exact user-facing string. All three are decisions somebody would otherwise make silently.\n\n## The numbers matter more than the prose\n\nThe strongest criterion is a worked example with real values. `₹2,400 with SAVE10 gives ₹2,160` is checkable by a human in two seconds, encodes the rounding rule, and cannot be satisfied by code that misunderstands the requirement.\n\nCompare with \"applies a 10% discount\", which is satisfied by code that discounts before tax, after tax, on shipping too, or that rounds half-up when your finance team rounds half-even. The worked example resolves all of those without anybody having to think of them as questions.\n\nThis matters more than it used to for a specific reason. A test whose expected value was derived from the implementation asserts whatever the implementation does. A worked example in the specification comes from outside the code, which is the only thing that makes an assertion evidence.\n\n## Criteria become test names, nearly verbatim\n\n```python\ndef test_promo_applies_before_expiry():        ...\ndef test_promo_rejected_after_expiry():        ...\ndef test_promo_not_applied_twice():            ...\ndef test_promo_discount_line_text():           ...\n```\n\nRun `pytest --collect-only -q` and you have the specification back, in a form that is executable and that will fail when somebody breaks it. That is what makes the criteria non-rotting: they are checked on every commit, and a criterion nobody checks is a wish.\n\nThe mapping should be close enough to be boring. If a criterion cannot be turned into a test name, it is usually not a criterion: it is an aspiration, and it needs another pass.\n\n## The criteria that cannot be automated\n\nBe honest that some cannot. \"The error message makes sense to a non-technical user.\" \"The screen is usable one-handed.\" \"The wording does not sound accusatory.\"\n\nThese are real requirements and they need a named human and a moment in the process, not a test. Write them in a separate section headed *checked by a person*, with a name against each. What you must not do is either drop them because they are not automatable, or pretend a test covers them.\n\n## The definition-of-done question\n\nOne more line, and it is the one that catches the largest omissions:\n\n> What must be true, other than the code working, before this is finished?\n\nUsually: the migration has run on production, the flag exists, the dashboard shows the new metric, the runbook mentions the new failure mode, the support team knows the new error message, the old code path is deleted.\n\nThat list is where features go to sit at 95% complete for three weeks. Writing it at the start does not make it shorter. It makes it visible, and visible work gets scheduled."},"check":{"question":"A ticket says \"apply a 10% discount\". The implementation discounts the subtotal before tax; finance expected the discount after tax. Both readings pass every test that was written. What would have prevented this?","options":[{"text":"A code review by someone from finance, since the ambiguity is a domain question rather than a technical one.","correct":false,"tempting":"Domain review helps, but it puts the check after the code is written when the cheap fix is one worked example beforehand."},{"text":"Property tests asserting the discount never exceeds the subtotal, which would have constrained the order of operations.","correct":false,"tempting":"That property holds under both readings, which is exactly why properties do not substitute for a specific expected value here."},{"text":"A criterion stating a specific input and its expected total.","correct":true},{"text":"Typed money values, so the pre-tax and post-tax amounts could not be confused at the call site.","correct":false,"tempting":"Branded types prevent mixing units and identifiers, but both readings here operate on the same currency type."}]},"prev":{"slug":"the-ambiguity-audit","title":"The ambiguity audit","url":"https://addaly.com/learn/how-software-changed/the-ambiguity-audit"},"next":{"slug":"decision-records","title":"Writing down the why","url":"https://addaly.com/learn/how-software-changed/decision-records"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Criteria that become test names', in How Software Development Changed. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/how-software-changed/acceptance-criteria-as-tests (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Criteria that become test names","courseTitle":"How Software Development Changed","module":{"index":4,"slug":"saying-what-you-want","title":"Saying what you want"},"answer":null,"keyIdea":"Replace every adjective in a requirement with an observation somebody could make, and write criteria as worked examples with real values, because a specification’s expected values are the only ones that come from outside the code.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"adjectives-are-not-criteria","runs":[{"text":"Adjectives are not criteria"}]},{"t":"p","runs":[{"text":"“The export should be fast.” “The form should be robust.” “Handle errors gracefully.” Each of these passes review in a ticket and specifies nothing, because every reader supplies a different meaning and nobody finds out until the disagreement is expensive."}]},{"t":"p","runs":[{"text":"The repair is mechanical. Replace each adjective with an observation somebody could make."}]},{"t":"ul","items":[[{"text":"Fast","i":true},{"text":" → 95th percentile under 300 ms at 50 requests per second, on the production dataset size."}],[{"text":"Robust","i":true},{"text":" → a malformed row does not abort the import; it is skipped, counted, and reported in the summary."}],[{"text":"Graceful","i":true},{"text":" → on a provider timeout, return 503 with "},{"text":"Retry-After: 30","code":true},{"text":", log at warning, do not retry a non-idempotent call."}],[{"text":"Secure","i":true},{"text":" → a user cannot read another tenant’s records through any endpoint in this module, tested with a second tenant’s token."}]]},{"t":"p","runs":[{"text":"Each rewrite took twenty seconds and each is now something a test can assert and a reviewer can check."}]},{"t":"h2","id":"the-given-when-then-shape","runs":[{"text":"The Given / When / Then shape"}]},{"t":"p","runs":[{"text":"One format, used consistently, does most of the work. It forces you to state the starting state, which is where the omissions live."}]},{"t":"code","lang":"","label":"Text","text":"Given a cart with 3 items totalling ₹2,400\nAnd a promo code SAVE10 valid until 30 September\nWhen the customer applies SAVE10 on 29 September\nThen the total becomes ₹2,160\nAnd the discount line reads \"SAVE10 −₹240\""},{"t":"p","runs":[{"text":"Then the negative cases, which is where the real specification lives:"}]},{"t":"code","lang":"","label":"Text","text":"Given the same cart\nWhen the customer applies SAVE10 on 1 October\nThen the code is rejected with \"This code has expired\"\nAnd the total is unchanged\n\nGiven a cart with SAVE10 already applied\nWhen the customer applies SAVE10 again\nThen the discount is not applied twice"},{"t":"p","runs":[{"text":"Three criteria and you have already decided expiry semantics, idempotence and the exact user-facing string. All three are decisions somebody would otherwise make silently."}]},{"t":"h2","id":"the-numbers-matter-more-than-the-prose","runs":[{"text":"The numbers matter more than the prose"}]},{"t":"p","runs":[{"text":"The strongest criterion is a worked example with real values. "},{"text":"₹2,400 with SAVE10 gives ₹2,160","code":true},{"text":" is checkable by a human in two seconds, encodes the rounding rule, and cannot be satisfied by code that misunderstands the requirement."}]},{"t":"p","runs":[{"text":"Compare with “applies a 10% discount”, which is satisfied by code that discounts before tax, after tax, on shipping too, or that rounds half-up when your finance team rounds half-even. The worked example resolves all of those without anybody having to think of them as questions."}]},{"t":"p","runs":[{"text":"This matters more than it used to for a specific reason. A test whose expected value was derived from the implementation asserts whatever the implementation does. A worked example in the specification comes from outside the code, which is the only thing that makes an assertion evidence."}]},{"t":"h2","id":"criteria-become-test-names-nearly-verbatim","runs":[{"text":"Criteria become test names, nearly verbatim"}]},{"t":"code","lang":"python","label":"Python","text":"def test_promo_applies_before_expiry():        ...\ndef test_promo_rejected_after_expiry():        ...\ndef test_promo_not_applied_twice():            ...\ndef test_promo_discount_line_text():           ..."},{"t":"p","runs":[{"text":"Run "},{"text":"pytest --collect-only -q","code":true},{"text":" and you have the specification back, in a form that is executable and that will fail when somebody breaks it. That is what makes the criteria non-rotting: they are checked on every commit, and a criterion nobody checks is a wish."}]},{"t":"p","runs":[{"text":"The mapping should be close enough to be boring. If a criterion cannot be turned into a test name, it is usually not a criterion: it is an aspiration, and it needs another pass."}]},{"t":"h2","id":"the-criteria-that-cannot-be-automated","runs":[{"text":"The criteria that cannot be automated"}]},{"t":"p","runs":[{"text":"Be honest that some cannot. “The error message makes sense to a non-technical user.” “The screen is usable one-handed.” “The wording does not sound accusatory.”"}]},{"t":"p","runs":[{"text":"These are real requirements and they need a named human and a moment in the process, not a test. Write them in a separate section headed "},{"text":"checked by a person","i":true},{"text":", with a name against each. What you must not do is either drop them because they are not automatable, or pretend a test covers them."}]},{"t":"h2","id":"the-definition-of-done-question","runs":[{"text":"The definition-of-done question"}]},{"t":"p","runs":[{"text":"One more line, and it is the one that catches the largest omissions:"}]},{"t":"quote","runs":[{"text":"What must be true, other than the code working, before this is finished?"}]},{"t":"p","runs":[{"text":"Usually: the migration has run on production, the flag exists, the dashboard shows the new metric, the runbook mentions the new failure mode, the support team knows the new error message, the old code path is deleted."}]},{"t":"p","runs":[{"text":"That list is where features go to sit at 95% complete for three weeks. Writing it at the start does not make it shorter. It makes it visible, and visible work gets scheduled."}]}],"check":{"question":[{"text":"A ticket says “apply a 10% discount”. The implementation discounts the subtotal before tax; finance expected the discount after tax. Both readings pass every test that was written. What would have prevented this?"}],"options":[{"text":[{"text":"A code review by someone from finance, since the ambiguity is a domain question rather than a technical one."}],"correct":false,"tempting":[{"text":"Domain review helps, but it puts the check after the code is written when the cheap fix is one worked example beforehand."}]},{"text":[{"text":"Property tests asserting the discount never exceeds the subtotal, which would have constrained the order of operations."}],"correct":false,"tempting":[{"text":"That property holds under both readings, which is exactly why properties do not substitute for a specific expected value here."}]},{"text":[{"text":"A criterion stating a specific input and its expected total."}],"correct":true,"tempting":null},{"text":[{"text":"Typed money values, so the pre-tax and post-tax amounts could not be confused at the call site."}],"correct":false,"tempting":[{"text":"Branded types prevent mixing units and identifiers, but both readings here operate on the same currency type."}]}],"order":[1,2,0,3],"why":null},"prev":{"slug":"the-ambiguity-audit","title":"The ambiguity audit","minutes":10},"next":{"slug":"decision-records","title":"Writing down the why","minutes":9}}}