{"course":{"slug":"cloud-and-shipping","title":"Shipping It","url":"https://addaly.com/learn/cloud-and-shipping","level":"intermediate","track":"build"},"lesson":{"slug":"processes-that-restart-themselves","title":"What keeps your process running when you close the laptop","url":"https://addaly.com/learn/cloud-and-shipping/processes-that-restart-themselves","position":3,"of":74,"minutes":9,"answer":"Something other than your terminal has to own the process: a supervisor that starts it at boot and restarts it on exit, plus the journal that says why it exited.","keyIdea":"Something other than your terminal has to own the process: a supervisor that starts it at boot and restarts it on exit, plus the journal that says why it exited, and a process that keeps vanishing without a log line was probably killed by the kernel for using too much memory."},"body":{"format":"markdown","text":"## The thing people do first, and why it fails\n\nThe first deployment nearly everybody performs is this: SSH in, run `npm start` or `python app.py`, see the site working, close the terminal. An hour later the site is down.\n\nIt is down because the process was a child of your login shell. When the connection dropped, it was sent a hangup signal and died. `nohup` and `tmux` fix that particular symptom, and people stop there, which is why so many small sites are one accidental `Ctrl-C`, one crash, or one reboot away from being offline until somebody notices.\n\nWhat you actually want is something whose job is to keep your process running: start it at boot, restart it when it exits, and record why it exited. On Linux that is `systemd`, already installed, and a unit file is about twelve lines.\n\n## A unit file, in full\n\n`/etc/systemd/system/myapp.service`:\n\n```ini\n[Unit]\nDescription=myapp\nAfter=network-online.target\n\n[Service]\nUser=myapp\nWorkingDirectory=/srv/myapp\nEnvironmentFile=/etc/myapp.env\nExecStart=/usr/bin/node /srv/myapp/server.js\nRestart=always\nRestartSec=2\n\n[Install]\nWantedBy=multi-user.target\n```\n\nThen:\n\n```bash\nsudo systemctl daemon-reload\nsudo systemctl enable --now myapp\nsystemctl status myapp\njournalctl -u myapp -f\n```\n\n`enable` is the part people forget: without it the service does not come back after a reboot. `Restart=always` restarts on any exit, including a clean one. `Restart=on-failure` restarts only on a non-zero exit, which is what you want if the process legitimately finishes.\n\nAnything your program writes to stdout and stderr goes to the journal, timestamped and searchable, with no logging library involved. `journalctl -u myapp --since \"10 min ago\"` is the fastest way to answer \"what did it say before it died\".\n\n## The restart loop that hides the bug\n\n`Restart=always` will happily restart a process that dies immediately, forever, several times a second. The site looks broken in a way that keeps changing, and the machine gets busy doing nothing.\n\nsystemd has a rate limit for this: by default, five starts within ten seconds and it gives up, leaving the unit `failed`. That is the correct behaviour, and it is worth knowing because \"the service will not start and systemctl says failed\" often means it started twenty times a minute ago. Tune with `StartLimitIntervalSec` and `StartLimitBurst`.\n\nA restart loop is a symptom, not a fix. The journal has the reason, usually in the last two lines before each restart: a missing environment variable, a port already in use, a database that is not up yet.\n\n## The killer nobody warns you about\n\nWhen a Linux machine runs out of memory, the kernel picks a process and kills it. Your application, being the biggest thing on a small box, is nearly always the one picked. It receives no signal it can catch, writes nothing to its own logs, and vanishes.\n\nThe evidence lives in the kernel log:\n\n```bash\ndmesg -T | grep -i -E 'killed process|out of memory'\njournalctl -k --since today | grep -i oom\n```\n\nIf you see it, the fix is memory (a smaller heap, fewer workers, a bigger box, or a leak found) not a bigger restart limit. Add swap only as a shock absorber; a machine that is swapping is a machine whose latency has already collapsed.\n\n## Containers do the same job with different words\n\nIn Docker, `--restart unless-stopped` is `Restart=always` with a different spelling. In Kubernetes, the restart policy plus liveness probes do it, and `CrashLoopBackOff` is the same rate limit as above, expressed as an escalating wait.\n\nOne container-specific trap: your process becomes PID 1, and PID 1 does not get the kernel's default signal handling. A process that ignores `SIGTERM` will sit there until the runtime kills it hard, ten seconds later, mid-request. Running with `docker run --init`, or handling `SIGTERM` yourself, avoids it.\n\n## The free paths\n\n`systemd` is free and already on the machine. `supervisord` is a good, simple alternative if you find unit files unpleasant. `pm2` is popular in Node projects and does restarts, log rotation and clustering, but note that it is one more thing that must itself be started at boot, usually by systemd, which is a hint about where the real answer lives.\n\nOn a managed platform (Fly, Render, Railway, Cloud Run, a Workers runtime) this lesson is done for you, and the platform's restart behaviour is worth reading once so you know what it does with a process that will not start."},"check":{"question":"Your service disappears every few days. There is nothing in the application log, no stack trace, and no error at all: the log simply stops mid-request and picks up again after a restart. Which explanation fits the evidence best?","options":[{"text":"An unhandled exception is crashing the process before the logger can flush","correct":false,"tempting":"Plausible, but an unhandled exception normally prints a stack trace to stderr, which the journal captures even when the application's own logger does not."},{"text":"The SSH session that started it was disconnected, sending a hangup signal","correct":false,"tempting":"The classic beginner failure, but it happens once, at the moment the terminal closes, not repeatedly on a service started by a supervisor."},{"text":"systemd hit its start rate limit and left the unit in a failed state","correct":false,"tempting":"Describes what happens after repeated fast restarts, not a cause of the first disappearance, and it would leave the service down, not running again."},{"text":"The kernel's OOM killer terminated the process, which gets no catchable signal and so writes nothing","correct":true}]},"prev":{"slug":"the-box-and-the-shell","title":"The first ten minutes on a new machine","url":"https://addaly.com/learn/cloud-and-shipping/the-box-and-the-shell"},"next":{"slug":"dns-and-domains","title":"Why your site is not loading","url":"https://addaly.com/learn/cloud-and-shipping/dns-and-domains"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'What keeps your process running when you close the laptop', in Shipping It. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/cloud-and-shipping/processes-that-restart-themselves (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"What keeps your process running when you close the laptop","courseTitle":"Shipping It","module":{"index":1,"slug":"how-the-internet-reaches-you","title":"How the internet reaches your process"},"answer":null,"keyIdea":"Something other than your terminal has to own the process: a supervisor that starts it at boot and restarts it on exit, plus the journal that says why it exited, and a process that keeps vanishing without a log line was probably killed by the kernel for using too much memory.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"the-thing-people-do-first-and-why-it-fails","runs":[{"text":"The thing people do first, and why it fails"}]},{"t":"p","runs":[{"text":"The first deployment nearly everybody performs is this: SSH in, run "},{"text":"npm start","code":true},{"text":" or "},{"text":"python app.py","code":true},{"text":", see the site working, close the terminal. An hour later the site is down."}]},{"t":"p","runs":[{"text":"It is down because the process was a child of your login shell. When the connection dropped, it was sent a hangup signal and died. "},{"text":"nohup","code":true},{"text":" and "},{"text":"tmux","code":true},{"text":" fix that particular symptom, and people stop there, which is why so many small sites are one accidental "},{"text":"Ctrl-C","code":true},{"text":", one crash, or one reboot away from being offline until somebody notices."}]},{"t":"p","runs":[{"text":"What you actually want is something whose job is to keep your process running: start it at boot, restart it when it exits, and record why it exited. On Linux that is "},{"text":"systemd","code":true},{"text":", already installed, and a unit file is about twelve lines."}]},{"t":"h2","id":"a-unit-file-in-full","runs":[{"text":"A unit file, in full"}]},{"t":"p","runs":[{"text":"/etc/systemd/system/myapp.service","code":true},{"text":":"}]},{"t":"code","lang":"ini","label":"INI","text":"[Unit]\nDescription=myapp\nAfter=network-online.target\n\n[Service]\nUser=myapp\nWorkingDirectory=/srv/myapp\nEnvironmentFile=/etc/myapp.env\nExecStart=/usr/bin/node /srv/myapp/server.js\nRestart=always\nRestartSec=2\n\n[Install]\nWantedBy=multi-user.target"},{"t":"p","runs":[{"text":"Then:"}]},{"t":"code","lang":"bash","label":"Bash","text":"sudo systemctl daemon-reload\nsudo systemctl enable --now myapp\nsystemctl status myapp\njournalctl -u myapp -f"},{"t":"p","runs":[{"text":"enable","code":true},{"text":" is the part people forget: without it the service does not come back after a reboot. "},{"text":"Restart=always","code":true},{"text":" restarts on any exit, including a clean one. "},{"text":"Restart=on-failure","code":true},{"text":" restarts only on a non-zero exit, which is what you want if the process legitimately finishes."}]},{"t":"p","runs":[{"text":"Anything your program writes to stdout and stderr goes to the journal, timestamped and searchable, with no logging library involved. "},{"text":"journalctl -u myapp --since \"10 min ago\"","code":true},{"text":" is the fastest way to answer “what did it say before it died”."}]},{"t":"h2","id":"the-restart-loop-that-hides-the-bug","runs":[{"text":"The restart loop that hides the bug"}]},{"t":"p","runs":[{"text":"Restart=always","code":true},{"text":" will happily restart a process that dies immediately, forever, several times a second. The site looks broken in a way that keeps changing, and the machine gets busy doing nothing."}]},{"t":"p","runs":[{"text":"systemd has a rate limit for this: by default, five starts within ten seconds and it gives up, leaving the unit "},{"text":"failed","code":true},{"text":". That is the correct behaviour, and it is worth knowing because “the service will not start and systemctl says failed” often means it started twenty times a minute ago. Tune with "},{"text":"StartLimitIntervalSec","code":true},{"text":" and "},{"text":"StartLimitBurst","code":true},{"text":"."}]},{"t":"p","runs":[{"text":"A restart loop is a symptom, not a fix. The journal has the reason, usually in the last two lines before each restart: a missing environment variable, a port already in use, a database that is not up yet."}]},{"t":"h2","id":"the-killer-nobody-warns-you-about","runs":[{"text":"The killer nobody warns you about"}]},{"t":"p","runs":[{"text":"When a Linux machine runs out of memory, the kernel picks a process and kills it. Your application, being the biggest thing on a small box, is nearly always the one picked. It receives no signal it can catch, writes nothing to its own logs, and vanishes."}]},{"t":"p","runs":[{"text":"The evidence lives in the kernel log:"}]},{"t":"code","lang":"bash","label":"Bash","text":"dmesg -T | grep -i -E 'killed process|out of memory'\njournalctl -k --since today | grep -i oom"},{"t":"p","runs":[{"text":"If you see it, the fix is memory (a smaller heap, fewer workers, a bigger box, or a leak found) not a bigger restart limit. Add swap only as a shock absorber; a machine that is swapping is a machine whose latency has already collapsed."}]},{"t":"h2","id":"containers-do-the-same-job-with-different-words","runs":[{"text":"Containers do the same job with different words"}]},{"t":"p","runs":[{"text":"In Docker, "},{"text":"--restart unless-stopped","code":true},{"text":" is "},{"text":"Restart=always","code":true},{"text":" with a different spelling. In Kubernetes, the restart policy plus liveness probes do it, and "},{"text":"CrashLoopBackOff","code":true},{"text":" is the same rate limit as above, expressed as an escalating wait."}]},{"t":"p","runs":[{"text":"One container-specific trap: your process becomes PID 1, and PID 1 does not get the kernel’s default signal handling. A process that ignores "},{"text":"SIGTERM","code":true},{"text":" will sit there until the runtime kills it hard, ten seconds later, mid-request. Running with "},{"text":"docker run --init","code":true},{"text":", or handling "},{"text":"SIGTERM","code":true},{"text":" yourself, avoids it."}]},{"t":"h2","id":"the-free-paths","runs":[{"text":"The free paths"}]},{"t":"p","runs":[{"text":"systemd","code":true},{"text":" is free and already on the machine. "},{"text":"supervisord","code":true},{"text":" is a good, simple alternative if you find unit files unpleasant. "},{"text":"pm2","code":true},{"text":" is popular in Node projects and does restarts, log rotation and clustering, but note that it is one more thing that must itself be started at boot, usually by systemd, which is a hint about where the real answer lives."}]},{"t":"p","runs":[{"text":"On a managed platform (Fly, Render, Railway, Cloud Run, a Workers runtime) this lesson is done for you, and the platform’s restart behaviour is worth reading once so you know what it does with a process that will not start."}]}],"check":{"question":[{"text":"Your service disappears every few days. There is nothing in the application log, no stack trace, and no error at all: the log simply stops mid-request and picks up again after a restart. Which explanation fits the evidence best?"}],"options":[{"text":[{"text":"An unhandled exception is crashing the process before the logger can flush"}],"correct":false,"tempting":[{"text":"Plausible, but an unhandled exception normally prints a stack trace to stderr, which the journal captures even when the application’s own logger does not."}]},{"text":[{"text":"The SSH session that started it was disconnected, sending a hangup signal"}],"correct":false,"tempting":[{"text":"The classic beginner failure, but it happens once, at the moment the terminal closes, not repeatedly on a service started by a supervisor."}]},{"text":[{"text":"systemd hit its start rate limit and left the unit in a failed state"}],"correct":false,"tempting":[{"text":"Describes what happens after repeated fast restarts, not a cause of the first disappearance, and it would leave the service down, not running again."}]},{"text":[{"text":"The kernel’s OOM killer terminated the process, which gets no catchable signal and so writes nothing"}],"correct":true,"tempting":null}],"order":[1,0,3,2],"why":null},"prev":{"slug":"the-box-and-the-shell","title":"The first ten minutes on a new machine","minutes":9},"next":{"slug":"dns-and-domains","title":"Why your site is not loading","minutes":8}}}