{"course":{"slug":"cloud-and-shipping","title":"Shipping It","url":"https://addaly.com/learn/cloud-and-shipping","level":"intermediate","track":"build"},"lesson":{"slug":"logs-at-2am","title":"Reading logs at 2am","url":"https://addaly.com/learn/cloud-and-shipping/logs-at-2am","position":20,"of":74,"minutes":7,"answer":"Give every request an ID and log it everywhere; that one habit turns guessing into searching.","keyIdea":"Give every request an ID and log it everywhere; that one habit turns guessing into searching."},"body":{"format":"markdown","text":"## Write for the person reading at 2am\n\nThat person is you, tired, on a phone, with none of the context you have right now. Every logging decision should be judged against that.\n\nHere is what most people ship:\n\n```\nerror [object Object]\n```\n\nHere is what helps:\n\n```json\n{\"level\":\"error\",\"msg\":\"thread create failed\",\"request_id\":\"01J8QF7X\",\n \"identity_id\":\"idn_82f\",\"room\":\"chai\",\"duration_ms\":30012,\n \"err\":\"connect ETIMEDOUT 10.0.3.4:5432\"}\n```\n\nOne line, one event, structured so you can filter on it. The duration alone tells you this was a 30-second timeout, which tells you the database was unreachable rather than slow.\n\n## The request ID is the highest-value habit\n\nGenerate an ID for every incoming request. Put it on every log line produced while handling that request. Return it in a response header. Show it on your error page: \"Something broke. Reference 01J8QF7X.\"\n\nA user sends you eight characters. You search for those eight characters and get the exact sequence of events, in order, for that one request among millions. Without it, you are filtering by timestamp and guessing which of the 400 requests in that second was theirs.\n\nMost frameworks give you this in a few lines. It is the cheapest thing in this course with the largest payoff.\n\n## Levels, used honestly\n\n- **error**: a human needs to look at this eventually.\n- **warn**: something degraded and was handled.\n- **info**: business events. Account created. Payment captured. Deploy started.\n- **debug**: off in production.\n\nThe common failure is not too few logs. It is an error that fires forty thousand times a day, that everyone has learned to scroll past. When everything is an error, nothing is. If a line is not actionable, it is not an error: demote it or delete it.\n\n## Never log\n\nPasswords. Tokens. Full card numbers. One-time codes. Whole request bodies containing personal data. The `Authorization` header. Logs get shipped to a search platform, retained for thirty days, and read by more people than can read your database.\n\n## The order of questions at 2am\n\n1. **Read the actual error string.** All of it. In a stack trace, find the first frame that is your code, not the framework's.\n2. **When did it start?** Then look at your deploy list. Most breakage is the most recent deploy, and rolling back is faster than diagnosing.\n3. **Is it everything or something?** Filter by route, by region, by user tier. \"All requests from one region\" and \"all requests to one route\" point at completely different causes.\n4. **Is it you or a dependency?** A timeout to a host that is not yours is not your bug, though it is still your outage.\n\n## Silence is a signal\n\nThis is the one worth carrying with you. If your load balancer is reporting 502s and your application logs show **nothing at all** for that window (no errors, and no successful requests either) the requests never reached your code. Your process crashed, failed to start, or is not listening on the port the platform expects.\n\nPeople lose an hour here because they read \"no errors in the logs\" as good news. An empty access log during a traffic spike is the loudest thing on the screen.\n\n## Logs are not monitoring\n\nLogs tell you what happened once you go looking. Something still has to make you look. A check that hits `/health` every minute and messages you when it fails is enough to begin with, and it takes about ten minutes to set up."},"check":{"question":"Users report checkout failing. Your load balancer shows a spike of 502s over a twelve-minute window. You filter application logs for that window and find nothing: no errors, and no request lines either. What does the silence most likely mean?","options":[{"text":"The requests never reached your application code, or the process died before it could write anything","correct":true},{"text":"The logging pipeline dropped lines under load; the app was handling and failing requests normally","correct":false,"tempting":"Log loss under pressure is a real phenomenon, so this reads as the cautious answer, but it explains the missing logs without explaining the 502s."},{"text":"The failures were logged at debug level, which is disabled in production","correct":false,"tempting":"Assumes a log-level misconfiguration, forgetting that even the successful request lines are missing."},{"text":"A 502 originates on the client side, so there would be nothing in server logs regardless","correct":false,"tempting":"Mixes up 4xx and 5xx, treating 'the browser saw an error' as 'the error belonged to the browser'."}]},"prev":{"slug":"rollback-and-the-kill-switch","title":"Undo, in under two minutes","url":"https://addaly.com/learn/cloud-and-shipping/rollback-and-the-kill-switch"},"next":{"slug":"where-logs-go-and-what-they-cost","title":"Where the logs go, and what they cost","url":"https://addaly.com/learn/cloud-and-shipping/where-logs-go-and-what-they-cost"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Reading logs at 2am', in Shipping It. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/cloud-and-shipping/logs-at-2am (Accessed: 31 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Reading logs at 2am","courseTitle":"Shipping It","module":{"index":3,"slug":"knowing-what-it-is-doing","title":"Knowing what it is doing"},"answer":null,"keyIdea":"Give every request an ID and log it everywhere; that one habit turns guessing into searching.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"write-for-the-person-reading-at-2am","runs":[{"text":"Write for the person reading at 2am"}]},{"t":"p","runs":[{"text":"That person is you, tired, on a phone, with none of the context you have right now. Every logging decision should be judged against that."}]},{"t":"p","runs":[{"text":"Here is what most people ship:"}]},{"t":"code","lang":"","label":"Text","text":"error [object Object]"},{"t":"p","runs":[{"text":"Here is what helps:"}]},{"t":"code","lang":"json","label":"JSON","text":"{\"level\":\"error\",\"msg\":\"thread create failed\",\"request_id\":\"01J8QF7X\",\n \"identity_id\":\"idn_82f\",\"room\":\"chai\",\"duration_ms\":30012,\n \"err\":\"connect ETIMEDOUT 10.0.3.4:5432\"}"},{"t":"p","runs":[{"text":"One line, one event, structured so you can filter on it. The duration alone tells you this was a 30-second timeout, which tells you the database was unreachable rather than slow."}]},{"t":"h2","id":"the-request-id-is-the-highest-value-habit","runs":[{"text":"The request ID is the highest-value habit"}]},{"t":"p","runs":[{"text":"Generate an ID for every incoming request. Put it on every log line produced while handling that request. Return it in a response header. Show it on your error page: “Something broke. Reference 01J8QF7X.”"}]},{"t":"p","runs":[{"text":"A user sends you eight characters. You search for those eight characters and get the exact sequence of events, in order, for that one request among millions. Without it, you are filtering by timestamp and guessing which of the 400 requests in that second was theirs."}]},{"t":"p","runs":[{"text":"Most frameworks give you this in a few lines. It is the cheapest thing in this course with the largest payoff."}]},{"t":"h2","id":"levels-used-honestly","runs":[{"text":"Levels, used honestly"}]},{"t":"ul","items":[[{"text":"error","b":true},{"text":": a human needs to look at this eventually."}],[{"text":"warn","b":true},{"text":": something degraded and was handled."}],[{"text":"info","b":true},{"text":": business events. Account created. Payment captured. Deploy started."}],[{"text":"debug","b":true},{"text":": off in production."}]]},{"t":"p","runs":[{"text":"The common failure is not too few logs. It is an error that fires forty thousand times a day, that everyone has learned to scroll past. When everything is an error, nothing is. If a line is not actionable, it is not an error: demote it or delete it."}]},{"t":"h2","id":"never-log","runs":[{"text":"Never log"}]},{"t":"p","runs":[{"text":"Passwords. Tokens. Full card numbers. One-time codes. Whole request bodies containing personal data. The "},{"text":"Authorization","code":true},{"text":" header. Logs get shipped to a search platform, retained for thirty days, and read by more people than can read your database."}]},{"t":"h2","id":"the-order-of-questions-at-2am","runs":[{"text":"The order of questions at 2am"}]},{"t":"ol","start":1,"items":[[{"text":"Read the actual error string.","b":true},{"text":" All of it. In a stack trace, find the first frame that is your code, not the framework’s."}],[{"text":"When did it start?","b":true},{"text":" Then look at your deploy list. Most breakage is the most recent deploy, and rolling back is faster than diagnosing."}],[{"text":"Is it everything or something?","b":true},{"text":" Filter by route, by region, by user tier. “All requests from one region” and “all requests to one route” point at completely different causes."}],[{"text":"Is it you or a dependency?","b":true},{"text":" A timeout to a host that is not yours is not your bug, though it is still your outage."}]]},{"t":"h2","id":"silence-is-a-signal","runs":[{"text":"Silence is a signal"}]},{"t":"p","runs":[{"text":"This is the one worth carrying with you. If your load balancer is reporting 502s and your application logs show "},{"text":"nothing at all","b":true},{"text":" for that window (no errors, and no successful requests either) the requests never reached your code. Your process crashed, failed to start, or is not listening on the port the platform expects."}]},{"t":"p","runs":[{"text":"People lose an hour here because they read “no errors in the logs” as good news. An empty access log during a traffic spike is the loudest thing on the screen."}]},{"t":"h2","id":"logs-are-not-monitoring","runs":[{"text":"Logs are not monitoring"}]},{"t":"p","runs":[{"text":"Logs tell you what happened once you go looking. Something still has to make you look. A check that hits "},{"text":"/health","code":true},{"text":" every minute and messages you when it fails is enough to begin with, and it takes about ten minutes to set up."}]}],"check":{"question":[{"text":"Users report checkout failing. Your load balancer shows a spike of 502s over a twelve-minute window. You filter application logs for that window and find nothing: no errors, and no request lines either. What does the silence most likely mean?"}],"options":[{"text":[{"text":"The requests never reached your application code, or the process died before it could write anything"}],"correct":true,"tempting":null},{"text":[{"text":"The logging pipeline dropped lines under load; the app was handling and failing requests normally"}],"correct":false,"tempting":[{"text":"Log loss under pressure is a real phenomenon, so this reads as the cautious answer, but it explains the missing logs without explaining the 502s."}]},{"text":[{"text":"The failures were logged at debug level, which is disabled in production"}],"correct":false,"tempting":[{"text":"Assumes a log-level misconfiguration, forgetting that even the successful request lines are missing."}]},{"text":[{"text":"A 502 originates on the client side, so there would be nothing in server logs regardless"}],"correct":false,"tempting":[{"text":"Mixes up 4xx and 5xx, treating ‘the browser saw an error’ as ‘the error belonged to the browser’."}]}],"order":[1,3,2,0],"why":null},"prev":{"slug":"rollback-and-the-kill-switch","title":"Undo, in under two minutes","minutes":9},"next":{"slug":"where-logs-go-and-what-they-cost","title":"Where the logs go, and what they cost","minutes":9}}}