{"course":{"slug":"cloud-and-shipping","title":"Shipping It","url":"https://addaly.com/learn/cloud-and-shipping","level":"intermediate","track":"build"},"lesson":{"slug":"caching-the-cheapest-server","title":"The request you never serve","url":"https://addaly.com/learn/cloud-and-shipping/caching-the-cheapest-server","position":30,"of":74,"minutes":9,"answer":"Caching converts traffic into nothing at all, and the two ways it goes wrong are serving one person's data to another and every copy expiring at the same instant.","keyIdea":"Caching converts traffic into nothing at all, and the two ways it goes wrong are serving one person's data to another and every copy expiring at the same instant."},"body":{"format":"markdown","text":"## The arithmetic\n\nA page gets 100 requests a second. You cache it at the CDN for 60 seconds. Your origin now serves **one request a minute** instead of 6,000. That is a 99.98% reduction, achieved with one header, on hardware you already pay for.\n\nNo amount of code optimisation competes with not running the code. Caching is the first thing to reach for when load or cost is the problem, and it is the thing most people reach for fourth.\n\n## The four places a response can be cached\n\n1. **The browser.** Free, closest to the user, and you cannot clear it.\n2. **The CDN.** Shared across users, purgeable, geographically close. This is where most of the win is.\n3. **Your application.** Redis, Valkey, or an in-process map. For computed results and database rows.\n4. **The database.** Its own page cache, which you influence but do not control.\n\nDifferent tools, one idea: keep the answer near whoever asks next.\n\n> Figure: Four places a response can be cached. Layers, top to bottom: Browser (free, closest to the user, and you cannot clear it); CDN (shared across users, purgeable, nearby; s-maxage and stale-while-revalidate live here); Your application (Redis, Valkey or an in-process map for computed results and rows); The database (its own page cache, which you influence but do not control). Keep the answer near whoever asks next. Most of the win is at the CDN, where one copy serves everybody for sixty seconds and the origin sees one request a minute instead of six thousand.\n\n## Two headers do most of the work\n\n**`Cache-Control`** states the policy. Learn four values:\n\n- Hashed static assets: `app.4f3a9c.js`, whose name changes when the content does:\n  `Cache-Control: public, max-age=31536000, immutable`\n  A year, and `immutable` tells the browser not to even revalidate on refresh.\n- HTML that everyone sees the same:\n  `Cache-Control: public, max-age=0, s-maxage=60, stale-while-revalidate=600`\n  Browsers revalidate every time; the CDN (`s-maxage`) serves a copy for a minute and may serve a stale copy for ten more while it fetches a fresh one behind the scenes. That last clause is the difference between a slow page after expiry and a fast one.\n- Anything personalised:\n  `Cache-Control: private, no-store`\n- Something that may be cached but must be checked:\n  `Cache-Control: no-cache`, which, confusingly, does not mean \"do not cache\". It means \"cache, but revalidate before reuse\". `no-store` is the one that means do not keep it.\n\n**`ETag`** is a fingerprint. The browser sends it back as `If-None-Match`; if it still matches you return `304 Not Modified` with no body. You still pay the round trip, but not the bytes. Useful for large responses that rarely change.\n\n## The bug that leaks one user's data to another\n\nThis is the dangerous one and it is easy to write.\n\nA page renders \"Welcome back, Priya\" and is served with `Cache-Control: public, max-age=300`. The CDN stores it. The next visitor is greeted as Priya, and if the page contains her order history, they can read it.\n\nThree rules that prevent it:\n\n- **Never mark a response `public` if it varies by session.** If the request had a session cookie and the response differs because of it, it is `private, no-store`.\n- **The cache key must include everything the response varies on.** If a page differs by language, the key needs the language: either in the URL or via a `Vary` header. `Vary: Cookie` is technically correct and practically useless, because every cookie value becomes a separate entry and your hit rate goes to zero. Put the varying thing in the path instead: `/en/pricing`, `/hi/pricing`.\n- **Watch for `Set-Cookie` on a cacheable response.** A cached response carrying somebody's session cookie hands their session to the next visitor. Most CDNs refuse to cache responses with `Set-Cookie` for exactly this reason: do not defeat that default.\n\n## Invalidation: change the name, do not purge\n\nPurging is eventually consistent, rate-limited, and awkward at scale. The alternative is to make the URL contain a content hash, so a new version is a new URL and the old one stops being requested. This is what every bundler does with `app.4f3a9c.js`, and it is why static assets can be cached for a year without risk.\n\nFor data caches, the same trick works: put a version number in the key. `user:8412:v3`. Bumping the version invalidates everything without deleting anything.\n\n## The thundering herd\n\nA popular key expires. In the same millisecond, 10,000 requests miss, and all 10,000 go to the database to compute the same value. The database falls over. Then it recovers, the key expires again, and it happens again.\n\nTwo fixes, both small:\n\n- **Single-flight.** Only the first request computes; the rest wait for its result. Most cache libraries have this; it is a lock around the recompute.\n- **Jitter the TTL.** Instead of 300 seconds, use 300 plus a random 0-60. Now the expiries spread out instead of aligning. One line, and it prevents the synchronised version of this problem entirely.\n\n## The free path\n\nCloudflare's free plan caches static assets globally and will cache HTML if you tell it to: that alone is a CDN most projects never outgrow. `nginx`'s `proxy_cache` and Varnish are free and run on your own box. Redis and Valkey are free. Every one of these is a one-afternoon addition, and any of them will do more for your latency and your bill than a bigger server."},"check":{"question":"A logged-in dashboard page is served with `Cache-Control: public, max-age=300` and sits behind a CDN. What is the failure you should expect?","options":[{"text":"One user's cached page is served to the next visitor, exposing their data","correct":true},{"text":"The page becomes stale, so users see data up to five minutes old","correct":false,"tempting":"The visible symptom people plan for, and much less serious than the one that actually occurs first with a `public` directive on a session-varying response."},{"text":"The CDN refuses to cache it because the request carried a session cookie","correct":false,"tempting":"Many CDNs do apply that default, but the question is what to expect from the header as written; and relying on the CDN's default to save you is not a design."},{"text":"Cache hit rates stay near zero because each user has a different cookie","correct":false,"tempting":"Describes what happens with `Vary: Cookie`, which is not present here; without it the cookie is not part of the key at all."}]},"prev":{"slug":"the-bill","title":"The bill, and the traffic that runs it up","url":"https://addaly.com/learn/cloud-and-shipping/the-bill"},"next":{"slug":"the-edge-and-the-cdn","title":"Serving from near the user, and what bandwidth costs","url":"https://addaly.com/learn/cloud-and-shipping/the-edge-and-the-cdn"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'The request you never serve', in Shipping It. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/cloud-and-shipping/caching-the-cheapest-server (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"The request you never serve","courseTitle":"Shipping It","module":{"index":4,"slug":"load-cost-and-abuse","title":"Load, cost and the traffic that is not customers"},"answer":null,"keyIdea":"Caching converts traffic into nothing at all, and the two ways it goes wrong are serving one person’s data to another and every copy expiring at the same instant.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"stack","title":"Four places a response can be cached","layers":[{"label":"Browser","detail":"free, closest to the user, and you cannot clear it"},{"label":"CDN","detail":"shared across users, purgeable, nearby; s-maxage and stale-while-revalidate live here"},{"label":"Your application","detail":"Redis, Valkey or an in-process map for computed results and rows"},{"label":"The database","detail":"its own page cache, which you influence but do not control"}],"caption":"Keep the answer near whoever asks next. Most of the win is at the CDN, where one copy serves everybody for sixty seconds and the origin sees one request a minute instead of six thousand."},"alt":"Figure: Four places a response can be cached. Layers, top to bottom: Browser (free, closest to the user, and you cannot clear it); CDN (shared across users, purgeable, nearby; s-maxage and stale-while-revalidate live here); Your application (Redis, Valkey or an in-process map for computed results and rows); The database (its own page cache, which you influence but do not control). Keep the answer near whoever asks next. Most of the win is at the CDN, where one copy serves everybody for sixty seconds and the origin sees one request a minute instead of six thousand."}},"blocks":[{"t":"h2","id":"the-arithmetic","runs":[{"text":"The arithmetic"}]},{"t":"p","runs":[{"text":"A page gets 100 requests a second. You cache it at the CDN for 60 seconds. Your origin now serves "},{"text":"one request a minute","b":true},{"text":" instead of 6,000. That is a 99.98% reduction, achieved with one header, on hardware you already pay for."}]},{"t":"p","runs":[{"text":"No amount of code optimisation competes with not running the code. Caching is the first thing to reach for when load or cost is the problem, and it is the thing most people reach for fourth."}]},{"t":"h2","id":"the-four-places-a-response-can-be-cached","runs":[{"text":"The four places a response can be cached"}]},{"t":"ol","start":1,"items":[[{"text":"The browser.","b":true},{"text":" Free, closest to the user, and you cannot clear it."}],[{"text":"The CDN.","b":true},{"text":" Shared across users, purgeable, geographically close. This is where most of the win is."}],[{"text":"Your application.","b":true},{"text":" Redis, Valkey, or an in-process map. For computed results and database rows."}],[{"text":"The database.","b":true},{"text":" Its own page cache, which you influence but do not control."}]]},{"t":"p","runs":[{"text":"Different tools, one idea: keep the answer near whoever asks next."}]},{"t":"h2","id":"two-headers-do-most-of-the-work","runs":[{"text":"Two headers do most of the work"}]},{"t":"p","runs":[{"text":"Cache-Control","b":true,"code":true},{"text":" states the policy. Learn four values:"}]},{"t":"ul","items":[[{"text":"Hashed static assets: "},{"text":"app.4f3a9c.js","code":true},{"text":", whose name changes when the content does:"}]]},{"t":"p","runs":[{"text":"Cache-Control: public, max-age=31536000, immutable","code":true},{"text":" A year, and "},{"text":"immutable","code":true},{"text":" tells the browser not to even revalidate on refresh."}]},{"t":"ul","items":[[{"text":"HTML that everyone sees the same:"}]]},{"t":"p","runs":[{"text":"Cache-Control: public, max-age=0, s-maxage=60, stale-while-revalidate=600","code":true},{"text":" Browsers revalidate every time; the CDN ("},{"text":"s-maxage","code":true},{"text":") serves a copy for a minute and may serve a stale copy for ten more while it fetches a fresh one behind the scenes. That last clause is the difference between a slow page after expiry and a fast one."}]},{"t":"ul","items":[[{"text":"Anything personalised:"}]]},{"t":"p","runs":[{"text":"Cache-Control: private, no-store","code":true}]},{"t":"ul","items":[[{"text":"Something that may be cached but must be checked:"}]]},{"t":"p","runs":[{"text":"Cache-Control: no-cache","code":true},{"text":", which, confusingly, does not mean “do not cache”. It means “cache, but revalidate before reuse”. "},{"text":"no-store","code":true},{"text":" is the one that means do not keep it."}]},{"t":"p","runs":[{"text":"ETag","b":true,"code":true},{"text":" is a fingerprint. The browser sends it back as "},{"text":"If-None-Match","code":true},{"text":"; if it still matches you return "},{"text":"304 Not Modified","code":true},{"text":" with no body. You still pay the round trip, but not the bytes. Useful for large responses that rarely change."}]},{"t":"h2","id":"the-bug-that-leaks-one-user-s-data-to-another","runs":[{"text":"The bug that leaks one user’s data to another"}]},{"t":"p","runs":[{"text":"This is the dangerous one and it is easy to write."}]},{"t":"p","runs":[{"text":"A page renders “Welcome back, Priya” and is served with "},{"text":"Cache-Control: public, max-age=300","code":true},{"text":". The CDN stores it. The next visitor is greeted as Priya, and if the page contains her order history, they can read it."}]},{"t":"p","runs":[{"text":"Three rules that prevent it:"}]},{"t":"ul","items":[[{"text":"Never mark a response ","b":true},{"text":"public","b":true,"code":true},{"text":" if it varies by session.","b":true},{"text":" If the request had a session cookie and the response differs because of it, it is "},{"text":"private, no-store","code":true},{"text":"."}],[{"text":"The cache key must include everything the response varies on.","b":true},{"text":" If a page differs by language, the key needs the language: either in the URL or via a "},{"text":"Vary","code":true},{"text":" header. "},{"text":"Vary: Cookie","code":true},{"text":" is technically correct and practically useless, because every cookie value becomes a separate entry and your hit rate goes to zero. Put the varying thing in the path instead: "},{"text":"/en/pricing","code":true},{"text":", "},{"text":"/hi/pricing","code":true},{"text":"."}],[{"text":"Watch for ","b":true},{"text":"Set-Cookie","b":true,"code":true},{"text":" on a cacheable response.","b":true},{"text":" A cached response carrying somebody’s session cookie hands their session to the next visitor. Most CDNs refuse to cache responses with "},{"text":"Set-Cookie","code":true},{"text":" for exactly this reason: do not defeat that default."}]]},{"t":"h2","id":"invalidation-change-the-name-do-not-purge","runs":[{"text":"Invalidation: change the name, do not purge"}]},{"t":"p","runs":[{"text":"Purging is eventually consistent, rate-limited, and awkward at scale. The alternative is to make the URL contain a content hash, so a new version is a new URL and the old one stops being requested. This is what every bundler does with "},{"text":"app.4f3a9c.js","code":true},{"text":", and it is why static assets can be cached for a year without risk."}]},{"t":"p","runs":[{"text":"For data caches, the same trick works: put a version number in the key. "},{"text":"user:8412:v3","code":true},{"text":". Bumping the version invalidates everything without deleting anything."}]},{"t":"h2","id":"the-thundering-herd","runs":[{"text":"The thundering herd"}]},{"t":"p","runs":[{"text":"A popular key expires. In the same millisecond, 10,000 requests miss, and all 10,000 go to the database to compute the same value. The database falls over. Then it recovers, the key expires again, and it happens again."}]},{"t":"p","runs":[{"text":"Two fixes, both small:"}]},{"t":"ul","items":[[{"text":"Single-flight.","b":true},{"text":" Only the first request computes; the rest wait for its result. Most cache libraries have this; it is a lock around the recompute."}],[{"text":"Jitter the TTL.","b":true},{"text":" Instead of 300 seconds, use 300 plus a random 0-60. Now the expiries spread out instead of aligning. One line, and it prevents the synchronised version of this problem entirely."}]]},{"t":"h2","id":"the-free-path","runs":[{"text":"The free path"}],"role":"free"},{"t":"p","runs":[{"text":"Cloudflare’s free plan caches static assets globally and will cache HTML if you tell it to: that alone is a CDN most projects never outgrow. "},{"text":"nginx","code":true},{"text":"’s "},{"text":"proxy_cache","code":true},{"text":" and Varnish are free and run on your own box. Redis and Valkey are free. Every one of these is a one-afternoon addition, and any of them will do more for your latency and your bill than a bigger server."}]}],"check":{"question":[{"text":"A logged-in dashboard page is served with "},{"text":"Cache-Control: public, max-age=300","code":true},{"text":" and sits behind a CDN. What is the failure you should expect?"}],"options":[{"text":[{"text":"One user’s cached page is served to the next visitor, exposing their data"}],"correct":true,"tempting":null},{"text":[{"text":"The page becomes stale, so users see data up to five minutes old"}],"correct":false,"tempting":[{"text":"The visible symptom people plan for, and much less serious than the one that actually occurs first with a "},{"text":"public","code":true},{"text":" directive on a session-varying response."}]},{"text":[{"text":"The CDN refuses to cache it because the request carried a session cookie"}],"correct":false,"tempting":[{"text":"Many CDNs do apply that default, but the question is what to expect from the header as written; and relying on the CDN’s default to save you is not a design."}]},{"text":[{"text":"Cache hit rates stay near zero because each user has a different cookie"}],"correct":false,"tempting":[{"text":"Describes what happens with "},{"text":"Vary: Cookie","code":true},{"text":", which is not present here; without it the cookie is not part of the key at all."}]}],"order":[2,0,1,3],"why":null},"prev":{"slug":"the-bill","title":"The bill, and the traffic that runs it up","minutes":9},"next":{"slug":"the-edge-and-the-cdn","title":"Serving from near the user, and what bandwidth costs","minutes":9}}}