{"course":{"slug":"cloud-and-shipping","title":"Shipping It","url":"https://addaly.com/learn/cloud-and-shipping","level":"intermediate","track":"build"},"lesson":{"slug":"keeping-the-connection-open","title":"The handshake you keep paying for","url":"https://addaly.com/learn/cloud-and-shipping/keeping-the-connection-open","position":9,"of":74,"minutes":9,"answer":"Most of a small request is handshake, so the wins are reuse: keep-alive in, a pooled client out, and an app idle timeout longer than the proxy's, or you will serve 502s nobody can explain.","keyIdea":"Most of a small request is handshake, so the wins are reuse: keep-alive in, a pooled client out, and an app idle timeout longer than the proxy's, or you will serve 502s nobody can explain."},"body":{"format":"markdown","text":"## Setting up a connection costs more than using it\n\nBefore a single byte of your response moves, a new HTTPS connection costs: one round trip for the TCP handshake, then one or two more for the TLS handshake. On a 40 ms link that is 80 to 120 ms spent agreeing to talk. Your handler might take 5 ms.\n\nThis is why connection reuse is one of the largest, cheapest wins available to a small site, and why the same mistake shows up in three different places.\n\n## Keep-alive, and the header that turns it off\n\nHTTP/1.1 connections are persistent by default: the browser sends a request, gets a response, and holds the socket open for the next one. `Connection: close` ends it. Some frameworks, proxies and old tutorials still set that header, and the effect is a fresh handshake per request, which on a page with 30 assets is 30 handshakes.\n\nOn the server side there is a timeout for idle keep-alive connections, usually 5 to 75 seconds. Two things about it matter:\n\n- Each held-open connection occupies a slot. A server configured for 200 connections with a 75-second idle timeout can be full of people who left.\n- If your load balancer's idle timeout is *longer* than your app's, the app will close a connection at the moment the balancer is handing it a request, and you get a small, steady drizzle of 502s that nothing in your code explains. Make the app's timeout the longer of the two. This is one of the most-reported mystery errors behind a proxy.\n\n## HTTP/2 and HTTP/3\n\nHTTP/2 sends many requests down one connection at once, which removes the old six-connections-per-host limit and makes handshake cost almost irrelevant for a browser. In practice you get it by turning it on at your proxy or CDN; it is one line of configuration and free.\n\nIt has one real weakness: everything is on one TCP connection, so a single lost packet stalls every stream on it until the retransmission arrives. That is head-of-line blocking, and it is worst on flaky mobile networks, exactly the networks much of this audience is on.\n\nHTTP/3 moves the whole thing to QUIC over UDP, where each stream is independent, so a lost packet affects one stream. It also merges the transport and TLS handshakes into one round trip. On a good fixed line the difference is small; on a congested mobile network it is not.\n\nOne caution repeated from the security literature: the 0-RTT resumption mode, which sends data before the handshake completes, is replayable by an attacker. Use it for GETs, never for anything that changes state.\n\n## The bug in your own outbound calls\n\nEverything above is about traffic coming in. The version that actually costs small services money is outgoing.\n\nIf your code calls a payment provider, a mail API or your own second service with a fresh client per request, you pay TCP plus TLS every time. Under load you also exhaust ephemeral ports and accumulate sockets in `TIME_WAIT`.\n\n```python\n# every call: new connection, new TLS handshake\nrequests.get(url)\n\n# one connection, reused\nsession = requests.Session()   # create once, at module scope\nsession.get(url)\n```\n\nThe same fix in other languages: a single `http.Client` in Go, one `HttpClient` in .NET, Node's `keepAlive` agent (on by default since Node 19, off before it), a connection pool in your HTTP library of choice. Measure it with:\n\n```bash\nss -s                       # totals, including TIME_WAIT\nss -tn state time-wait | wc -l\ncurl -w '%{time_connect} %{time_appconnect} %{time_total}\\n' -o /dev/null -s https://example.com\n```\n\nRun that `curl` twice in a row against your own API. The second run does not get faster on its own, but comparing `time_connect` and `time_appconnect` tells you exactly how much of your latency is handshake, and therefore how much reuse would remove.\n\n## Where this shows up as a bill\n\nSome providers charge per connection or per new TLS session; more commonly, handshakes show up as CPU. TLS termination is not free, and on a small instance a few hundred new handshakes a second is measurable. Reuse and a CDN in front are the two ways out, in that order of effort.\n\n## What to do with this\n\n1. Confirm keep-alive is on and no proxy is sending `Connection: close`.\n2. Make your app's idle timeout longer than the load balancer's.\n3. Turn on HTTP/2 at the edge; add HTTP/3 if your CDN offers it, which is usually a checkbox.\n4. Find every place your code makes an outbound HTTP call and give it one shared, pooled client.\n\nThe fourth item is where the surprise usually is."},"check":{"question":"Your app sits behind a load balancer with a 60-second idle timeout; the app's own keep-alive timeout is 5 seconds. Users see occasional 502s with nothing in the application log. What is the mechanism?","options":[{"text":"The load balancer reuses a pooled connection at the moment the app is closing it, so the request is lost in flight","correct":true},{"text":"The app is running out of file descriptors because the balancer holds connections open for 60 seconds","correct":false,"tempting":"Descriptor exhaustion is real but produces accept failures and errors in the app's own log, not silent 502s from the layer above."},{"text":"TLS session resumption is failing, so some handshakes are rejected by the balancer","correct":false,"tempting":"Resumption failure costs a round trip; it does not produce 502s, which are the balancer reporting a bad response from upstream."},{"text":"The app is too slow and the balancer is timing the request out","correct":false,"tempting":"A request timeout usually presents as 504 and correlates with load; these errors appear on idle connections, independent of traffic."}]},"prev":{"slug":"one-request-end-to-end","title":"One request, with a stopwatch","url":"https://addaly.com/learn/cloud-and-shipping/one-request-end-to-end"},"next":{"slug":"secrets-and-env","title":"The key you must not commit","url":"https://addaly.com/learn/cloud-and-shipping/secrets-and-env"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The handshake you keep paying for', in Shipping It. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/cloud-and-shipping/keeping-the-connection-open (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The handshake you keep paying for","courseTitle":"Shipping It","module":{"index":1,"slug":"how-the-internet-reaches-you","title":"How the internet reaches your process"},"answer":null,"keyIdea":"Most of a small request is handshake, so the wins are reuse: keep-alive in, a pooled client out, and an app idle timeout longer than the proxy’s, or you will serve 502s nobody can explain.","hero":{"kind":"plate","image":null,"alt":null,"caption":null},"blocks":[{"t":"h2","id":"setting-up-a-connection-costs-more-than-using-it","runs":[{"text":"Setting up a connection costs more than using it"}]},{"t":"p","runs":[{"text":"Before a single byte of your response moves, a new HTTPS connection costs: one round trip for the TCP handshake, then one or two more for the TLS handshake. On a 40 ms link that is 80 to 120 ms spent agreeing to talk. Your handler might take 5 ms."}]},{"t":"p","runs":[{"text":"This is why connection reuse is one of the largest, cheapest wins available to a small site, and why the same mistake shows up in three different places."}]},{"t":"h2","id":"keep-alive-and-the-header-that-turns-it-off","runs":[{"text":"Keep-alive, and the header that turns it off"}]},{"t":"p","runs":[{"text":"HTTP/1.1 connections are persistent by default: the browser sends a request, gets a response, and holds the socket open for the next one. "},{"text":"Connection: close","code":true},{"text":" ends it. Some frameworks, proxies and old tutorials still set that header, and the effect is a fresh handshake per request, which on a page with 30 assets is 30 handshakes."}]},{"t":"p","runs":[{"text":"On the server side there is a timeout for idle keep-alive connections, usually 5 to 75 seconds. Two things about it matter:"}]},{"t":"ul","items":[[{"text":"Each held-open connection occupies a slot. A server configured for 200 connections with a 75-second idle timeout can be full of people who left."}],[{"text":"If your load balancer’s idle timeout is "},{"text":"longer","i":true},{"text":" than your app’s, the app will close a connection at the moment the balancer is handing it a request, and you get a small, steady drizzle of 502s that nothing in your code explains. Make the app’s timeout the longer of the two. This is one of the most-reported mystery errors behind a proxy."}]]},{"t":"h2","id":"http-2-and-http-3","runs":[{"text":"HTTP/2 and HTTP/3"}]},{"t":"p","runs":[{"text":"HTTP/2 sends many requests down one connection at once, which removes the old six-connections-per-host limit and makes handshake cost almost irrelevant for a browser. In practice you get it by turning it on at your proxy or CDN; it is one line of configuration and free."}]},{"t":"p","runs":[{"text":"It has one real weakness: everything is on one TCP connection, so a single lost packet stalls every stream on it until the retransmission arrives. That is head-of-line blocking, and it is worst on flaky mobile networks, exactly the networks much of this audience is on."}]},{"t":"p","runs":[{"text":"HTTP/3 moves the whole thing to QUIC over UDP, where each stream is independent, so a lost packet affects one stream. It also merges the transport and TLS handshakes into one round trip. On a good fixed line the difference is small; on a congested mobile network it is not."}]},{"t":"p","runs":[{"text":"One caution repeated from the security literature: the 0-RTT resumption mode, which sends data before the handshake completes, is replayable by an attacker. Use it for GETs, never for anything that changes state."}]},{"t":"h2","id":"the-bug-in-your-own-outbound-calls","runs":[{"text":"The bug in your own outbound calls"}]},{"t":"p","runs":[{"text":"Everything above is about traffic coming in. The version that actually costs small services money is outgoing."}]},{"t":"p","runs":[{"text":"If your code calls a payment provider, a mail API or your own second service with a fresh client per request, you pay TCP plus TLS every time. Under load you also exhaust ephemeral ports and accumulate sockets in "},{"text":"TIME_WAIT","code":true},{"text":"."}]},{"t":"code","lang":"python","label":"Python","text":"# every call: new connection, new TLS handshake\nrequests.get(url)\n\n# one connection, reused\nsession = requests.Session()   # create once, at module scope\nsession.get(url)"},{"t":"p","runs":[{"text":"The same fix in other languages: a single "},{"text":"http.Client","code":true},{"text":" in Go, one "},{"text":"HttpClient","code":true},{"text":" in .NET, Node’s "},{"text":"keepAlive","code":true},{"text":" agent (on by default since Node 19, off before it), a connection pool in your HTTP library of choice. Measure it with:"}]},{"t":"code","lang":"bash","label":"Bash","text":"ss -s                       # totals, including TIME_WAIT\nss -tn state time-wait | wc -l\ncurl -w '%{time_connect} %{time_appconnect} %{time_total}\\n' -o /dev/null -s https://example.com"},{"t":"p","runs":[{"text":"Run that "},{"text":"curl","code":true},{"text":" twice in a row against your own API. The second run does not get faster on its own, but comparing "},{"text":"time_connect","code":true},{"text":" and "},{"text":"time_appconnect","code":true},{"text":" tells you exactly how much of your latency is handshake, and therefore how much reuse would remove."}]},{"t":"h2","id":"where-this-shows-up-as-a-bill","runs":[{"text":"Where this shows up as a bill"}]},{"t":"p","runs":[{"text":"Some providers charge per connection or per new TLS session; more commonly, handshakes show up as CPU. TLS termination is not free, and on a small instance a few hundred new handshakes a second is measurable. Reuse and a CDN in front are the two ways out, in that order of effort."}]},{"t":"h2","id":"what-to-do-with-this","runs":[{"text":"What to do with this"}]},{"t":"ol","start":1,"items":[[{"text":"Confirm keep-alive is on and no proxy is sending "},{"text":"Connection: close","code":true},{"text":"."}],[{"text":"Make your app’s idle timeout longer than the load balancer’s."}],[{"text":"Turn on HTTP/2 at the edge; add HTTP/3 if your CDN offers it, which is usually a checkbox."}],[{"text":"Find every place your code makes an outbound HTTP call and give it one shared, pooled client."}]]},{"t":"p","runs":[{"text":"The fourth item is where the surprise usually is."}]}],"check":{"question":[{"text":"Your app sits behind a load balancer with a 60-second idle timeout; the app’s own keep-alive timeout is 5 seconds. Users see occasional 502s with nothing in the application log. What is the mechanism?"}],"options":[{"text":[{"text":"The load balancer reuses a pooled connection at the moment the app is closing it, so the request is lost in flight"}],"correct":true,"tempting":null},{"text":[{"text":"The app is running out of file descriptors because the balancer holds connections open for 60 seconds"}],"correct":false,"tempting":[{"text":"Descriptor exhaustion is real but produces accept failures and errors in the app’s own log, not silent 502s from the layer above."}]},{"text":[{"text":"TLS session resumption is failing, so some handshakes are rejected by the balancer"}],"correct":false,"tempting":[{"text":"Resumption failure costs a round trip; it does not produce 502s, which are the balancer reporting a bad response from upstream."}]},{"text":[{"text":"The app is too slow and the balancer is timing the request out"}],"correct":false,"tempting":[{"text":"A request timeout usually presents as 504 and correlates with load; these errors appear on idle connections, independent of traffic."}]}],"order":[1,2,0,3],"why":null},"prev":{"slug":"one-request-end-to-end","title":"One request, with a stopwatch","minutes":10},"next":{"slug":"secrets-and-env","title":"The key you must not commit","minutes":8}}}