Reading logs at 2am

Write for the person reading at 2am

That person is you, tired, on a phone, with none of the context you have right now. Every logging decision should be judged against that.

Here is what most people ship:

error [object Object]

Here is what helps:

{"level":"error","msg":"thread create failed","request_id":"01J8QF7X",
 "identity_id":"idn_82f","room":"chai","duration_ms":30012,
 "err":"connect ETIMEDOUT 10.0.3.4:5432"}

One line, one event, structured so you can filter on it. The duration alone tells you this was a 30-second timeout, which tells you the database was unreachable rather than slow.

The request ID is the highest-value habit

Generate an ID for every incoming request. Put it on every log line produced while handling that request. Return it in a response header. Show it on your error page: "Something broke. Reference 01J8QF7X."

A user sends you eight characters. You search for those eight characters and get the exact sequence of events, in order, for that one request among millions. Without it, you are filtering by timestamp and guessing which of the 400 requests in that second was theirs.

Most frameworks give you this in a few lines. It is the cheapest thing in this course with the largest payoff.

Levels, used honestly

• error — a human needs to look at this eventually.
• warn — something degraded and was handled.
• info — business events. Account created. Payment captured. Deploy started.
• debug — off in production.

The common failure is not too few logs. It is an error that fires forty thousand times a day, that everyone has learned to scroll past. When everything is an error, nothing is. If a line is not actionable, it is not an error — demote it or delete it.

Never log

Passwords. Tokens. Full card numbers. One-time codes. Whole request bodies containing personal data. The Authorization header. Logs get shipped to a search platform, retained for thirty days, and read by more people than can read your database.

The order of questions at 2am

1. 1Read the actual error string. All of it. In a stack trace, find the first frame that is your code, not the framework's.
2. 2When did it start? Then look at your deploy list. Most breakage is the most recent deploy, and rolling back is faster than diagnosing.
3. 3Is it everything or something? Filter by route, by region, by user tier. "All requests from one region" and "all requests to one route" point at completely different causes.
4. 4Is it you or a dependency? A timeout to a host that is not yours is not your bug, though it is still your outage.

Silence is a signal

This is the one worth carrying with you. If your load balancer is reporting 502s and your application logs show nothing at all for that window — no errors, and no successful requests either — the requests never reached your code. Your process crashed, failed to start, or is not listening on the port the platform expects.

People lose an hour here because they read "no errors in the logs" as good news. An empty access log during a traffic spike is the loudest thing on the screen.

Logs are not monitoring

Logs tell you what happened once you go looking. Something still has to make you look. A check that hits /health every minute and messages you when it fails is enough to begin with, and it takes about ten minutes to set up.