{"course":{"slug":"cloud-and-shipping","title":"Shipping It","url":"https://addaly.com/learn/cloud-and-shipping","level":"intermediate","track":"build"},"lesson":{"slug":"stopping-a-process-politely","title":"Ending a process without dropping a request","url":"https://addaly.com/learn/cloud-and-shipping/stopping-a-process-politely","position":16,"of":74,"minutes":8,"answer":"A deploy stops a process as well as starting one, so catch SIGTERM, fail readiness first, drain with a deadline shorter than the platform's grace period.","keyIdea":"A deploy stops a process as well as starting one, so catch SIGTERM, fail readiness first, drain with a deadline shorter than the platform's grace period, and check that your container is not swallowing the signal at PID 1."},"body":{"format":"markdown","text":"## Every deploy kills something\n\nA deploy is two events: a new process starts, and an old process stops. Almost all the attention goes to the first. The dropped requests come from the second.\n\nWhen the old process is told to stop, it is usually in the middle of something: three HTTP requests half-answered, a database transaction open, a background job two seconds into a five-second task, a websocket holding somebody's live page. What it does in the next few seconds decides whether your users see anything at all.\n\n## The two signals\n\n`SIGTERM` is a polite request to stop. It can be caught, and that is the point: your code gets to decide what happens.\n\n`SIGKILL` cannot be caught. The process ends immediately, mid-write, mid-request.\n\nEvery deploy system does the same thing: send `SIGTERM`, wait a grace period, then send `SIGKILL`. The grace period is 10 seconds in Docker and Kubernetes by default, 90 in systemd. If your requests take longer than the grace period, some of them are being cut off on every deploy, and the only sign is a small bump in 502s that looks like noise.\n\n## What graceful shutdown actually means\n\nFour steps, in this order:\n\n1. **Fail the readiness check first.** Start reporting not-ready so the load balancer stops sending new requests. This must happen *before* you stop accepting, and you have to wait: health checks poll every few seconds, so there is a window where the balancer still believes in you. Sleeping 5 seconds here removes more dropped requests than anything else in this lesson.\n2. **Stop accepting new connections**, keep serving the ones in flight.\n3. **Finish what is open**, with a deadline.\n4. **Close the database pool and exit.**\n\nIn Node:\n\n```js\nprocess.on('SIGTERM', async () => {\n  await setTimeout(5000);            // let the LB notice we are unready\n  server.close(async () => {          // stop accepting, drain in-flight\n    await pool.end();\n    process.exit(0);\n  });\n  setTimeout(() => process.exit(1), 25_000).unref();  // hard deadline\n});\n```\n\nThe last line matters. A drain with no deadline is a process that never exits, and the platform will `SIGKILL` it anyway, later, and less predictably. Make your own timeout shorter than the platform's grace period, and set the platform's grace period longer than your slowest normal request.\n\n> Figure: Graceful shutdown, in the order that drops nothing. Steps: 1. SIGTERM arrives (a polite request; PID 1 in a container must actually catch it); 2. Fail readiness (return 503 so the load balancer stops sending new work); 3. Wait about 5 s (health checks poll; the balancer has to notice); 4. Stop accepting, finish in-flight (server.close() with a deadline of your own); 5. Close the pool and exit 0 (before the platform's 10 s (Docker) or 90 s (systemd) runs out). The five seconds of doing nothing after failing readiness are the most valuable five seconds in the deploy. Your own deadline must be shorter than the platform's grace period, or SIGKILL arrives mid-request anyway.\n\n## The trap in containers\n\nIn a container your process is PID 1, and PID 1 does not get default signal handling from the kernel. If it has no `SIGTERM` handler, the signal is ignored entirely and the container always dies by `SIGKILL` after the grace period.\n\nWorse, if your image starts the app through a shell (`CMD npm start`, or an `ENTRYPOINT` written as a shell string) the shell is PID 1 and does not forward signals to its child. Your handler is there and never runs.\n\nTwo fixes: use the exec form (`CMD [\"node\", \"server.js\"]`), and run with `docker run --init` or `tini` so a real init process sits at PID 1 and forwards signals.\n\nThis is worth testing directly, because it is invisible otherwise:\n\n```bash\ndocker run --rm --name t myapp &\ntime docker stop t      # ~0.2s means it handled SIGTERM; ~10s means it did not\n```\n\nThat single command has caught this bug in more projects than any amount of reading.\n\n## Background workers need the same thing, differently\n\nA web request lasts milliseconds; a job might last minutes. Draining a worker means: stop pulling new jobs, finish the current one, then exit. If the current job is too long for the grace period, the right design is a job that checks a stop flag between steps and re-queues its remainder, which is only safe if the job is idempotent, which is the argument made in the queues lesson.\n\nSchedulers and cron-like runners need one more: on a hard kill, the lock a job was holding must expire on its own, or the job never runs again.\n\n## What to check on your own service\n\n- Does `SIGTERM` reach your code at all? Log a line in the handler and stop the process to see it.\n- Is your drain deadline shorter than the platform grace period?\n- Do you fail readiness before you stop accepting, with a pause in between?\n- Does a deploy under a small load test produce zero failed requests? Run one with `oha` or `hey`, both free, and deploy while it runs. That is the only proof."},"check":{"question":"Your app has a tested SIGTERM handler, yet `docker stop` always takes the full ten seconds and requests are still cut off. The Dockerfile ends with `CMD npm start`. What is happening?","options":[{"text":"The shell started by `CMD` is PID 1 and does not forward SIGTERM to the Node process, so the handler never runs","correct":true},{"text":"Ten seconds is the drain deadline in the handler, so it is working as written","correct":false,"tempting":"Would produce a clean exit at ten seconds with no cut-off requests; the dropped requests show the handler is not draining at all."},{"text":"Docker sends SIGKILL first for containers whose main process is a package manager","correct":false,"tempting":"Invents a special case. Docker always sends SIGTERM first regardless of what the process is."},{"text":"The load balancer is still routing traffic because readiness has not failed","correct":false,"tempting":"A real cause of dropped requests in general, but it would not explain why stopping the container takes the full grace period every time."}]},"prev":{"slug":"deploying-small","title":"Deploying for nearly nothing","url":"https://addaly.com/learn/cloud-and-shipping/deploying-small"},"next":{"slug":"staging-and-preview","title":"The copy you are allowed to break","url":"https://addaly.com/learn/cloud-and-shipping/staging-and-preview"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Ending a process without dropping a request', in Shipping It. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/cloud-and-shipping/stopping-a-process-politely (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Ending a process without dropping a request","courseTitle":"Shipping It","module":{"index":2,"slug":"making-a-deploy-boring","title":"Making a deploy boring"},"answer":null,"keyIdea":"A deploy stops a process as well as starting one, so catch SIGTERM, fail readiness first, drain with a deadline shorter than the platform’s grace period, and check that your container is not swallowing the signal at PID 1.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"flow","steps":[{"label":"SIGTERM arrives","detail":"a polite request; PID 1 in a container must actually catch it"},{"label":"Fail readiness","detail":"return 503 so the load balancer stops sending new work"},{"label":"Wait about 5 s","detail":"health checks poll; the balancer has to notice"},{"label":"Stop accepting, finish in-flight","detail":"server.close() with a deadline of your own"},{"label":"Close the pool and exit 0","detail":"before the platform’s 10 s (Docker) or 90 s (systemd) runs out"}],"title":"Graceful shutdown, in the order that drops nothing","caption":"The five seconds of doing nothing after failing readiness are the most valuable five seconds in the deploy. Your own deadline must be shorter than the platform’s grace period, or SIGKILL arrives mid-request anyway."},"alt":"Figure: Graceful shutdown, in the order that drops nothing. Steps: 1. SIGTERM arrives (a polite request; PID 1 in a container must actually catch it); 2. Fail readiness (return 503 so the load balancer stops sending new work); 3. Wait about 5 s (health checks poll; the balancer has to notice); 4. Stop accepting, finish in-flight (server.close() with a deadline of your own); 5. Close the pool and exit 0 (before the platform’s 10 s (Docker) or 90 s (systemd) runs out). The five seconds of doing nothing after failing readiness are the most valuable five seconds in the deploy. Your own deadline must be shorter than the platform’s grace period, or SIGKILL arrives mid-request anyway."}},"blocks":[{"t":"h2","id":"every-deploy-kills-something","runs":[{"text":"Every deploy kills something"}]},{"t":"p","runs":[{"text":"A deploy is two events: a new process starts, and an old process stops. Almost all the attention goes to the first. The dropped requests come from the second."}]},{"t":"p","runs":[{"text":"When the old process is told to stop, it is usually in the middle of something: three HTTP requests half-answered, a database transaction open, a background job two seconds into a five-second task, a websocket holding somebody’s live page. What it does in the next few seconds decides whether your users see anything at all."}]},{"t":"h2","id":"the-two-signals","runs":[{"text":"The two signals"}]},{"t":"p","runs":[{"text":"SIGTERM","code":true},{"text":" is a polite request to stop. It can be caught, and that is the point: your code gets to decide what happens."}]},{"t":"p","runs":[{"text":"SIGKILL","code":true},{"text":" cannot be caught. The process ends immediately, mid-write, mid-request."}]},{"t":"p","runs":[{"text":"Every deploy system does the same thing: send "},{"text":"SIGTERM","code":true},{"text":", wait a grace period, then send "},{"text":"SIGKILL","code":true},{"text":". The grace period is 10 seconds in Docker and Kubernetes by default, 90 in systemd. If your requests take longer than the grace period, some of them are being cut off on every deploy, and the only sign is a small bump in 502s that looks like noise."}]},{"t":"h2","id":"what-graceful-shutdown-actually-means","runs":[{"text":"What graceful shutdown actually means"}]},{"t":"p","runs":[{"text":"Four steps, in this order:"}]},{"t":"ol","start":1,"items":[[{"text":"Fail the readiness check first.","b":true},{"text":" Start reporting not-ready so the load balancer stops sending new requests. This must happen "},{"text":"before","i":true},{"text":" you stop accepting, and you have to wait: health checks poll every few seconds, so there is a window where the balancer still believes in you. Sleeping 5 seconds here removes more dropped requests than anything else in this lesson."}],[{"text":"Stop accepting new connections","b":true},{"text":", keep serving the ones in flight."}],[{"text":"Finish what is open","b":true},{"text":", with a deadline."}],[{"text":"Close the database pool and exit.","b":true}]]},{"t":"p","runs":[{"text":"In Node:"}]},{"t":"code","lang":"js","label":"JavaScript","text":"process.on('SIGTERM', async () => {\n  await setTimeout(5000);            // let the LB notice we are unready\n  server.close(async () => {          // stop accepting, drain in-flight\n    await pool.end();\n    process.exit(0);\n  });\n  setTimeout(() => process.exit(1), 25_000).unref();  // hard deadline\n});"},{"t":"p","runs":[{"text":"The last line matters. A drain with no deadline is a process that never exits, and the platform will "},{"text":"SIGKILL","code":true},{"text":" it anyway, later, and less predictably. Make your own timeout shorter than the platform’s grace period, and set the platform’s grace period longer than your slowest normal request."}]},{"t":"h2","id":"the-trap-in-containers","runs":[{"text":"The trap in containers"}]},{"t":"p","runs":[{"text":"In a container your process is PID 1, and PID 1 does not get default signal handling from the kernel. If it has no "},{"text":"SIGTERM","code":true},{"text":" handler, the signal is ignored entirely and the container always dies by "},{"text":"SIGKILL","code":true},{"text":" after the grace period."}]},{"t":"p","runs":[{"text":"Worse, if your image starts the app through a shell ("},{"text":"CMD npm start","code":true},{"text":", or an "},{"text":"ENTRYPOINT","code":true},{"text":" written as a shell string) the shell is PID 1 and does not forward signals to its child. Your handler is there and never runs."}]},{"t":"p","runs":[{"text":"Two fixes: use the exec form ("},{"text":"CMD [\"node\", \"server.js\"]","code":true},{"text":"), and run with "},{"text":"docker run --init","code":true},{"text":" or "},{"text":"tini","code":true},{"text":" so a real init process sits at PID 1 and forwards signals."}]},{"t":"p","runs":[{"text":"This is worth testing directly, because it is invisible otherwise:"}]},{"t":"code","lang":"bash","label":"Bash","text":"docker run --rm --name t myapp &\ntime docker stop t      # ~0.2s means it handled SIGTERM; ~10s means it did not"},{"t":"p","runs":[{"text":"That single command has caught this bug in more projects than any amount of reading."}]},{"t":"h2","id":"background-workers-need-the-same-thing-differently","runs":[{"text":"Background workers need the same thing, differently"}]},{"t":"p","runs":[{"text":"A web request lasts milliseconds; a job might last minutes. Draining a worker means: stop pulling new jobs, finish the current one, then exit. If the current job is too long for the grace period, the right design is a job that checks a stop flag between steps and re-queues its remainder, which is only safe if the job is idempotent, which is the argument made in the queues lesson."}]},{"t":"p","runs":[{"text":"Schedulers and cron-like runners need one more: on a hard kill, the lock a job was holding must expire on its own, or the job never runs again."}]},{"t":"h2","id":"what-to-check-on-your-own-service","runs":[{"text":"What to check on your own service"}]},{"t":"ul","items":[[{"text":"Does "},{"text":"SIGTERM","code":true},{"text":" reach your code at all? Log a line in the handler and stop the process to see it."}],[{"text":"Is your drain deadline shorter than the platform grace period?"}],[{"text":"Do you fail readiness before you stop accepting, with a pause in between?"}],[{"text":"Does a deploy under a small load test produce zero failed requests? Run one with "},{"text":"oha","code":true},{"text":" or "},{"text":"hey","code":true},{"text":", both free, and deploy while it runs. That is the only proof."}]]}],"check":{"question":[{"text":"Your app has a tested SIGTERM handler, yet "},{"text":"docker stop","code":true},{"text":" always takes the full ten seconds and requests are still cut off. The Dockerfile ends with "},{"text":"CMD npm start","code":true},{"text":". What is happening?"}],"options":[{"text":[{"text":"The shell started by "},{"text":"CMD","code":true},{"text":" is PID 1 and does not forward SIGTERM to the Node process, so the handler never runs"}],"correct":true,"tempting":null},{"text":[{"text":"Ten seconds is the drain deadline in the handler, so it is working as written"}],"correct":false,"tempting":[{"text":"Would produce a clean exit at ten seconds with no cut-off requests; the dropped requests show the handler is not draining at all."}]},{"text":[{"text":"Docker sends SIGKILL first for containers whose main process is a package manager"}],"correct":false,"tempting":[{"text":"Invents a special case. Docker always sends SIGTERM first regardless of what the process is."}]},{"text":[{"text":"The load balancer is still routing traffic because readiness has not failed"}],"correct":false,"tempting":[{"text":"A real cause of dropped requests in general, but it would not explain why stopping the container takes the full grace period every time."}]}],"order":[1,0,2,3],"why":null},"prev":{"slug":"deploying-small","title":"Deploying for nearly nothing","minutes":8},"next":{"slug":"staging-and-preview","title":"The copy you are allowed to break","minutes":8}}}