{"course":{"slug":"cloud-and-shipping","title":"Shipping It","url":"https://addaly.com/learn/cloud-and-shipping","level":"intermediate","track":"build"},"lesson":{"slug":"health-checks-that-tell-the-truth","title":"The health check that lies","url":"https://addaly.com/learn/cloud-and-shipping/health-checks-that-tell-the-truth","position":25,"of":74,"minutes":9,"answer":"A health check that always returns 200 tests only that your web server can serve a constant, and the check you rely on must run from outside the thing it is checking.","keyIdea":"A health check that always returns 200 tests only that your web server can serve a constant, and the check you rely on must run from outside the thing it is checking."},"body":{"format":"markdown","text":"## The check that proves nothing\n\n```js\napp.get(\"/health\", (req, res) => res.json({ ok: true }));\n```\n\nThis tells you that the process is running and the HTTP layer works. It will return 200 while the database is unreachable, the disk is full, the cache is down and every real request is failing.\n\nIt is not useless (a load balancer needs something cheap to poll) but it must not be the thing you trust when someone asks whether the site is up.\n\n## Liveness and readiness are different questions\n\n- **Liveness: should I be restarted?** Answer no unless the process is wedged: a deadlock, an exhausted event loop. Keep it trivial.\n- **Readiness: should I receive traffic right now?** Answer no while you are still warming up, while migrations are running, or while your database connection is broken.\n\nGetting them the wrong way round is a classic and it is spectacular. Put a database check in your *liveness* probe and a database blip restarts every container at once, which drops every connection, which makes the database worse, which fails the next probe. A restart loop across your whole fleet, caused by a monitoring decision.\n\nRule: **liveness restarts, readiness routes.** Only readiness gets to depend on anything outside the process.\n\n## A deep check that does not become the problem\n\nA useful readiness check touches the dependencies you cannot work without:\n\n```js\napp.get(\"/ready\", async (req, res) => {\n  const checks = await Promise.allSettled([\n    withTimeout(db.query(\"select 1\"), 800),\n    withTimeout(cache.ping(), 300),\n  ]);\n  const ok = checks.every((c) => c.status === \"fulfilled\");\n  res.status(ok ? 200 : 503).json({ ok, checks: summarise(checks) });\n});\n```\n\nTwo details that are not optional. **Timeouts** on every dependency, or the check hangs instead of failing and the platform learns nothing. And **cache the result for a few seconds**, because a probe every second across ten instances means ten extra database queries a second forever, and when the database is struggling, the health checks pile on and finish it off. A health check that makes an overloaded system worse is a feedback loop, not a diagnostic.\n\nReturn 503, not 200-with-a-false-field. Infrastructure reads the status code.\n\n## Your own monitoring runs on the machine that is down\n\nThis is the sentence to remember. If your health check is polled from inside the same box, the same cluster, or the same region, it goes silent exactly when you need it, and silence is not an alert.\n\nSo you need at least one check that runs somewhere else entirely, on infrastructure you do not control, from more than one region.\n\nThe free options are real:\n\n- **UptimeRobot**: 50 monitors on the free plan, five-minute interval.\n- **Uptime Kuma**: open source, self-hosted, and if you host it on a different provider from your app it counts as outside.\n- **Better Stack** and **Cronitor** both have usable free tiers.\n- A GitHub Actions cron job that curls your site and opens an issue on failure. Crude, free, works.\n\n## Monitor the thing that matters, not the homepage\n\nA homepage is often static and cached, so it stays up while the application is dead. Check something that exercises the real path: log in as a test user and load a page that queries the database. That is a synthetic check, and one of them is worth twenty pings.\n\nWhile you are there, monitor the things from the module ahead of you too: TLS certificate expiry and domain expiry are one HTTP check each and they prevent two of the most embarrassing outages there are.\n\n## Say 503 during a deploy\n\nWhen you are shutting down, stop accepting new requests, finish the ones in flight, and answer readiness with 503 and a `Retry-After`. That is how a load balancer learns to stop sending you traffic *before* you disappear, which is the difference between a clean deploy and a handful of failed requests every time.\n\nThe signal handler is short and almost nobody writes it:\n\n```js\nprocess.on(\"SIGTERM\", async () => {\n  ready = false;             // readiness now returns 503\n  await sleep(5000);         // let the load balancer notice\n  server.close(() => process.exit(0));\n});\n```\n\nThose five seconds of doing nothing are the most valuable five seconds in your deploy."},"check":{"question":"A brief database hiccup causes every container in your fleet to restart simultaneously, and the restart loop continues after the database recovers. Which configuration mistake produces exactly this?","options":[{"text":"The readiness probe has no timeout, so it hangs instead of failing","correct":false,"tempting":"A real bug, but a hanging readiness probe removes instances from rotation; it does not order the platform to kill and restart them."},{"text":"The health check queries the database on every probe, adding load during the incident","correct":false,"tempting":"Genuinely makes an incident worse and is worth fixing, but extra queries do not cause restarts on their own."},{"text":"The database check is in the liveness probe, so a dependency failure is being treated as \"this process is wedged, kill it\"","correct":true},{"text":"The containers share a connection pool that was exhausted, so every instance failed at once","correct":false,"tempting":"Describes why the failure was simultaneous rather than why it became a restart loop, and the loop persisting after recovery points at the probe."}]},"prev":{"slug":"errors-that-find-you","title":"Errors that come to you","url":"https://addaly.com/learn/cloud-and-shipping/errors-that-find-you"},"next":{"slug":"alerts-worth-waking-for","title":"Alerts you would get out of bed for","url":"https://addaly.com/learn/cloud-and-shipping/alerts-worth-waking-for"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The health check that lies', in Shipping It. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/cloud-and-shipping/health-checks-that-tell-the-truth (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The health check that lies","courseTitle":"Shipping It","module":{"index":3,"slug":"knowing-what-it-is-doing","title":"Knowing what it is doing"},"answer":null,"keyIdea":"A health check that always returns 200 tests only that your web server can serve a constant, and the check you rely on must run from outside the thing it is checking.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-check-that-proves-nothing","runs":[{"text":"The check that proves nothing"}]},{"t":"code","lang":"js","label":"JavaScript","text":"app.get(\"/health\", (req, res) => res.json({ ok: true }));"},{"t":"p","runs":[{"text":"This tells you that the process is running and the HTTP layer works. It will return 200 while the database is unreachable, the disk is full, the cache is down and every real request is failing."}]},{"t":"p","runs":[{"text":"It is not useless (a load balancer needs something cheap to poll) but it must not be the thing you trust when someone asks whether the site is up."}]},{"t":"h2","id":"liveness-and-readiness-are-different-questions","runs":[{"text":"Liveness and readiness are different questions"}]},{"t":"ul","items":[[{"text":"Liveness: should I be restarted?","b":true},{"text":" Answer no unless the process is wedged: a deadlock, an exhausted event loop. Keep it trivial."}],[{"text":"Readiness: should I receive traffic right now?","b":true},{"text":" Answer no while you are still warming up, while migrations are running, or while your database connection is broken."}]]},{"t":"p","runs":[{"text":"Getting them the wrong way round is a classic and it is spectacular. Put a database check in your "},{"text":"liveness","i":true},{"text":" probe and a database blip restarts every container at once, which drops every connection, which makes the database worse, which fails the next probe. A restart loop across your whole fleet, caused by a monitoring decision."}]},{"t":"p","runs":[{"text":"Rule: "},{"text":"liveness restarts, readiness routes.","b":true},{"text":" Only readiness gets to depend on anything outside the process."}]},{"t":"h2","id":"a-deep-check-that-does-not-become-the-problem","runs":[{"text":"A deep check that does not become the problem"}]},{"t":"p","runs":[{"text":"A useful readiness check touches the dependencies you cannot work without:"}]},{"t":"code","lang":"js","label":"JavaScript","text":"app.get(\"/ready\", async (req, res) => {\n  const checks = await Promise.allSettled([\n    withTimeout(db.query(\"select 1\"), 800),\n    withTimeout(cache.ping(), 300),\n  ]);\n  const ok = checks.every((c) => c.status === \"fulfilled\");\n  res.status(ok ? 200 : 503).json({ ok, checks: summarise(checks) });\n});"},{"t":"p","runs":[{"text":"Two details that are not optional. "},{"text":"Timeouts","b":true},{"text":" on every dependency, or the check hangs instead of failing and the platform learns nothing. And "},{"text":"cache the result for a few seconds","b":true},{"text":", because a probe every second across ten instances means ten extra database queries a second forever, and when the database is struggling, the health checks pile on and finish it off. A health check that makes an overloaded system worse is a feedback loop, not a diagnostic."}]},{"t":"p","runs":[{"text":"Return 503, not 200-with-a-false-field. Infrastructure reads the status code."}]},{"t":"h2","id":"your-own-monitoring-runs-on-the-machine-that-is-down","runs":[{"text":"Your own monitoring runs on the machine that is down"}]},{"t":"p","runs":[{"text":"This is the sentence to remember. If your health check is polled from inside the same box, the same cluster, or the same region, it goes silent exactly when you need it, and silence is not an alert."}]},{"t":"p","runs":[{"text":"So you need at least one check that runs somewhere else entirely, on infrastructure you do not control, from more than one region."}]},{"t":"p","runs":[{"text":"The free options are real:"}]},{"t":"ul","items":[[{"text":"UptimeRobot","b":true},{"text":": 50 monitors on the free plan, five-minute interval."}],[{"text":"Uptime Kuma","b":true},{"text":": open source, self-hosted, and if you host it on a different provider from your app it counts as outside."}],[{"text":"Better Stack","b":true},{"text":" and "},{"text":"Cronitor","b":true},{"text":" both have usable free tiers."}],[{"text":"A GitHub Actions cron job that curls your site and opens an issue on failure. Crude, free, works."}]]},{"t":"h2","id":"monitor-the-thing-that-matters-not-the-homepage","runs":[{"text":"Monitor the thing that matters, not the homepage"}]},{"t":"p","runs":[{"text":"A homepage is often static and cached, so it stays up while the application is dead. Check something that exercises the real path: log in as a test user and load a page that queries the database. That is a synthetic check, and one of them is worth twenty pings."}]},{"t":"p","runs":[{"text":"While you are there, monitor the things from the module ahead of you too: TLS certificate expiry and domain expiry are one HTTP check each and they prevent two of the most embarrassing outages there are."}]},{"t":"h2","id":"say-503-during-a-deploy","runs":[{"text":"Say 503 during a deploy"}]},{"t":"p","runs":[{"text":"When you are shutting down, stop accepting new requests, finish the ones in flight, and answer readiness with 503 and a "},{"text":"Retry-After","code":true},{"text":". That is how a load balancer learns to stop sending you traffic "},{"text":"before","i":true},{"text":" you disappear, which is the difference between a clean deploy and a handful of failed requests every time."}]},{"t":"p","runs":[{"text":"The signal handler is short and almost nobody writes it:"}]},{"t":"code","lang":"js","label":"JavaScript","text":"process.on(\"SIGTERM\", async () => {\n  ready = false;             // readiness now returns 503\n  await sleep(5000);         // let the load balancer notice\n  server.close(() => process.exit(0));\n});"},{"t":"p","runs":[{"text":"Those five seconds of doing nothing are the most valuable five seconds in your deploy."}]}],"check":{"question":[{"text":"A brief database hiccup causes every container in your fleet to restart simultaneously, and the restart loop continues after the database recovers. Which configuration mistake produces exactly this?"}],"options":[{"text":[{"text":"The readiness probe has no timeout, so it hangs instead of failing"}],"correct":false,"tempting":[{"text":"A real bug, but a hanging readiness probe removes instances from rotation; it does not order the platform to kill and restart them."}]},{"text":[{"text":"The health check queries the database on every probe, adding load during the incident"}],"correct":false,"tempting":[{"text":"Genuinely makes an incident worse and is worth fixing, but extra queries do not cause restarts on their own."}]},{"text":[{"text":"The database check is in the liveness probe, so a dependency failure is being treated as “this process is wedged, kill it”"}],"correct":true,"tempting":null},{"text":[{"text":"The containers share a connection pool that was exhausted, so every instance failed at once"}],"correct":false,"tempting":[{"text":"Describes why the failure was simultaneous rather than why it became a restart loop, and the loop persisting after recovery points at the probe."}]}],"order":[0,1,2,3],"why":null},"prev":{"slug":"errors-that-find-you","title":"Errors that come to you","minutes":8},"next":{"slug":"alerts-worth-waking-for","title":"Alerts you would get out of bed for","minutes":8}}}