{"course":{"slug":"cloud-and-shipping","title":"Shipping It","url":"https://addaly.com/learn/cloud-and-shipping","level":"intermediate","track":"build"},"lesson":{"slug":"the-postmortem","title":"Writing down what happened","url":"https://addaly.com/learn/cloud-and-shipping/the-postmortem","position":46,"of":74,"minutes":9,"answer":"A postmortem is worth writing only if it changes something, so measure detection time, name systemic causes rather than people, and leave with fewer than four owned actions.","keyIdea":"A postmortem is worth writing only if it changes something, so measure detection time, name systemic causes rather than people, and leave with fewer than four owned actions."},"body":{"format":"markdown","text":"## The purpose is one thing\n\nNot a record. Not accountability. A postmortem exists to make the next incident less likely or less long. Everything in its structure should serve that, and anything that does not is ceremony.\n\n## Blameless means something specific\n\nIt does not mean pretending nobody ran the command. It means the question is **\"what made this easy to do and hard to notice?\"** rather than \"who did it?\"\n\nThe reasoning is practical rather than kind. In an organisation that punishes the person who caused an incident, people stop volunteering that they caused one. Detection time goes up, causes go unrecorded, and the same failure recurs with less information each time. Blamelessness is what keeps information flowing.\n\n\"Human error\" is never a root cause. It is a place to start asking. Why was that command available? Why did nothing check? Why did the interface make the destructive option adjacent to the safe one?\n\n## The structure\n\n**Summary.** Two sentences. What broke, for how long.\n\n**Impact, in user terms and in numbers.** Not \"some users saw errors\". Instead: *42 minutes, roughly 3,100 failed requests, 180 signups lost, checkout unavailable in Europe.* Numbers make the priority argument for you afterwards; adjectives do not.\n\n**Timeline with timestamps.** From your incident log. Include when it started, when it was **detected**, when the first action was taken, when it recovered.\n\n**What happened.** The mechanism, plainly.\n\n**Contributing factors.** Plural, always.\n\n**What went well.** Real, not decorative: the alert fired correctly, the rollback worked, the runbook was accurate. Knowing which defences worked tells you which ones to keep investing in.\n\n**Actions.** Owned and dated.\n\n## The most useful number is detection time\n\nIf the incident lasted 42 minutes and you learned about it at minute 31 from a customer, then the outage was 11 minutes of response and 31 minutes of ignorance. The thing to fix is detection, not the bug.\n\nTeams reliably fix the bug and leave the detection gap in place, which is why the next unrelated bug also runs for half an hour. Split the duration into detect, decide and fix, every time. It tells you where your effort belongs.\n\n> Figure: A 42-minute outage, split the useful way. Bars (minutes): Start until somebody noticed 31; Noticed until recovered 11. The bug was fixed in eleven minutes once anybody knew. The other thirty-one were ignorance, which means the action that shortens the next incident is detection, not the bug.\n\n## Five whys is not enough\n\nThe technique of asking \"why\" five times produces a single causal chain, and real incidents are not chains. Three things were true at once: the check did not exist, the alert threshold was wrong, and the runbook was stale. A chain picks one and calls it the root cause.\n\nAsk instead: **\"what else had to be true for this to happen?\"** at each step. That produces a set of contributing factors, and it is a much better guide to what to change, because fixing any one of them may have been enough.\n\n## Three actions, owned, dated\n\nA postmortem with fourteen unowned action items produces nothing at all, and worse, it teaches everyone that writing them is theatre. Pick the two or three that would most reduce recurrence or detection time. Give each a name and a date. Track them where your other work is tracked, not in the document.\n\nOne action that actually lands beats twelve that do not, by an infinite ratio.\n\n## Count them over a quarter\n\nIndividual postmortems fix individual incidents. Reading a quarter of them together shows the pattern, and the pattern is usually small: most teams find that three causes account for most of their incidents. That is a much better basis for deciding what to build than any individual document.\n\n## Even alone, write it\n\nA `docs/incidents/2027-01-14-login-outage.md` costs twenty minutes. Six months later it is the only record of why the odd guard clause exists, why that timeout is 3 seconds and not 30, and why the deploy script refuses to run on a dirty tree. Your future self is a different person with none of your current context, and this is the cheapest way to send them a message."},"check":{"question":"An outage lasted 42 minutes. It began at 14:02, a customer emailed at 14:33, you rolled back at 14:41 and it recovered at 14:44. The postmortem's single action item is a fix for the bug that caused it. What has been missed?","options":[{"text":"The timeline should include what each person was doing at each step","correct":false,"tempting":"Useful context, but it describes the document's completeness rather than a defect in what it concluded."},{"text":"Nothing was learned about detection: 31 of the 42 minutes were before anyone knew, and no alert existed for this failure","correct":true},{"text":"The impact should be quantified in failed requests and lost signups","correct":false,"tempting":"A genuine omission from the write-up, and it affects prioritisation later; but it does not change what the next incident will look like."},{"text":"The postmortem should assign responsibility for the change that introduced the bug","correct":false,"tempting":"The opposite of what a postmortem is for; it reduces what people report and makes future detection worse rather than better."}]},"prev":{"slug":"the-first-thirty-minutes","title":"The first thirty minutes of an outage","url":"https://addaly.com/learn/cloud-and-shipping/the-first-thirty-minutes"},"next":{"slug":"practising-failure","title":"Breaking it on purpose, on a Tuesday","url":"https://addaly.com/learn/cloud-and-shipping/practising-failure"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Writing down what happened', in Shipping It. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/cloud-and-shipping/the-postmortem (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Writing down what happened","courseTitle":"Shipping It","module":{"index":5,"slug":"staying-alive","title":"Staying alive: the bad day and the day after"},"answer":null,"keyIdea":"A postmortem is worth writing only if it changes something, so measure detection time, name systemic causes rather than people, and leave with fewer than four owned actions.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"bars","unit":"minutes","title":"A 42-minute outage, split the useful way","series":[{"label":"Start until somebody noticed","value":31,"highlight":true},{"label":"Noticed until recovered","value":11}],"caption":"The bug was fixed in eleven minutes once anybody knew. The other thirty-one were ignorance, which means the action that shortens the next incident is detection, not the bug."},"alt":"Figure: A 42-minute outage, split the useful way. Bars (minutes): Start until somebody noticed 31; Noticed until recovered 11. The bug was fixed in eleven minutes once anybody knew. The other thirty-one were ignorance, which means the action that shortens the next incident is detection, not the bug."}},"blocks":[{"t":"h2","id":"the-purpose-is-one-thing","runs":[{"text":"The purpose is one thing"}]},{"t":"p","runs":[{"text":"Not a record. Not accountability. A postmortem exists to make the next incident less likely or less long. Everything in its structure should serve that, and anything that does not is ceremony."}]},{"t":"h2","id":"blameless-means-something-specific","runs":[{"text":"Blameless means something specific"}]},{"t":"p","runs":[{"text":"It does not mean pretending nobody ran the command. It means the question is "},{"text":"”what made this easy to do and hard to notice?”","b":true},{"text":" rather than “who did it?”"}]},{"t":"p","runs":[{"text":"The reasoning is practical rather than kind. In an organisation that punishes the person who caused an incident, people stop volunteering that they caused one. Detection time goes up, causes go unrecorded, and the same failure recurs with less information each time. Blamelessness is what keeps information flowing."}]},{"t":"p","runs":[{"text":"“Human error” is never a root cause. It is a place to start asking. Why was that command available? Why did nothing check? Why did the interface make the destructive option adjacent to the safe one?"}]},{"t":"h2","id":"the-structure","runs":[{"text":"The structure"}]},{"t":"p","runs":[{"text":"Summary.","b":true},{"text":" Two sentences. What broke, for how long."}]},{"t":"p","runs":[{"text":"Impact, in user terms and in numbers.","b":true},{"text":" Not “some users saw errors”. Instead: "},{"text":"42 minutes, roughly 3,100 failed requests, 180 signups lost, checkout unavailable in Europe.","i":true},{"text":" Numbers make the priority argument for you afterwards; adjectives do not."}]},{"t":"p","runs":[{"text":"Timeline with timestamps.","b":true},{"text":" From your incident log. Include when it started, when it was "},{"text":"detected","b":true},{"text":", when the first action was taken, when it recovered."}]},{"t":"p","runs":[{"text":"What happened.","b":true},{"text":" The mechanism, plainly."}]},{"t":"p","runs":[{"text":"Contributing factors.","b":true},{"text":" Plural, always."}]},{"t":"p","runs":[{"text":"What went well.","b":true},{"text":" Real, not decorative: the alert fired correctly, the rollback worked, the runbook was accurate. Knowing which defences worked tells you which ones to keep investing in."}]},{"t":"p","runs":[{"text":"Actions.","b":true},{"text":" Owned and dated."}]},{"t":"h2","id":"the-most-useful-number-is-detection-time","runs":[{"text":"The most useful number is detection time"}]},{"t":"p","runs":[{"text":"If the incident lasted 42 minutes and you learned about it at minute 31 from a customer, then the outage was 11 minutes of response and 31 minutes of ignorance. The thing to fix is detection, not the bug."}]},{"t":"p","runs":[{"text":"Teams reliably fix the bug and leave the detection gap in place, which is why the next unrelated bug also runs for half an hour. Split the duration into detect, decide and fix, every time. It tells you where your effort belongs."}]},{"t":"h2","id":"five-whys-is-not-enough","runs":[{"text":"Five whys is not enough"}]},{"t":"p","runs":[{"text":"The technique of asking “why” five times produces a single causal chain, and real incidents are not chains. Three things were true at once: the check did not exist, the alert threshold was wrong, and the runbook was stale. A chain picks one and calls it the root cause."}]},{"t":"p","runs":[{"text":"Ask instead: "},{"text":"”what else had to be true for this to happen?”","b":true},{"text":" at each step. That produces a set of contributing factors, and it is a much better guide to what to change, because fixing any one of them may have been enough."}]},{"t":"h2","id":"three-actions-owned-dated","runs":[{"text":"Three actions, owned, dated"}]},{"t":"p","runs":[{"text":"A postmortem with fourteen unowned action items produces nothing at all, and worse, it teaches everyone that writing them is theatre. Pick the two or three that would most reduce recurrence or detection time. Give each a name and a date. Track them where your other work is tracked, not in the document."}]},{"t":"p","runs":[{"text":"One action that actually lands beats twelve that do not, by an infinite ratio."}]},{"t":"h2","id":"count-them-over-a-quarter","runs":[{"text":"Count them over a quarter"}]},{"t":"p","runs":[{"text":"Individual postmortems fix individual incidents. Reading a quarter of them together shows the pattern, and the pattern is usually small: most teams find that three causes account for most of their incidents. That is a much better basis for deciding what to build than any individual document."}]},{"t":"h2","id":"even-alone-write-it","runs":[{"text":"Even alone, write it"}]},{"t":"p","runs":[{"text":"A "},{"text":"docs/incidents/2027-01-14-login-outage.md","code":true},{"text":" costs twenty minutes. Six months later it is the only record of why the odd guard clause exists, why that timeout is 3 seconds and not 30, and why the deploy script refuses to run on a dirty tree. Your future self is a different person with none of your current context, and this is the cheapest way to send them a message."}]}],"check":{"question":[{"text":"An outage lasted 42 minutes. It began at 14:02, a customer emailed at 14:33, you rolled back at 14:41 and it recovered at 14:44. The postmortem’s single action item is a fix for the bug that caused it. What has been missed?"}],"options":[{"text":[{"text":"The timeline should include what each person was doing at each step"}],"correct":false,"tempting":[{"text":"Useful context, but it describes the document’s completeness rather than a defect in what it concluded."}]},{"text":[{"text":"Nothing was learned about detection: 31 of the 42 minutes were before anyone knew, and no alert existed for this failure"}],"correct":true,"tempting":null},{"text":[{"text":"The impact should be quantified in failed requests and lost signups"}],"correct":false,"tempting":[{"text":"A genuine omission from the write-up, and it affects prioritisation later; but it does not change what the next incident will look like."}]},{"text":[{"text":"The postmortem should assign responsibility for the change that introduced the bug"}],"correct":false,"tempting":[{"text":"The opposite of what a postmortem is for; it reduces what people report and makes future detection worse rather than better."}]}],"order":[3,1,2,0],"why":null},"prev":{"slug":"the-first-thirty-minutes","title":"The first thirty minutes of an outage","minutes":10},"next":{"slug":"practising-failure","title":"Breaking it on purpose, on a Tuesday","minutes":8}}}