The request you never serve

The arithmetic

A page gets 100 requests a second. You cache it at the CDN for 60 seconds. Your origin now serves one request a minute instead of 6,000. That is a 99.98% reduction, achieved with one header, on hardware you already pay for.

No amount of code optimisation competes with not running the code. Caching is the first thing to reach for when load or cost is the problem, and it is the thing most people reach for fourth.

The four places a response can be cached

1. The browser. Free, closest to the user, and you cannot clear it.
2. The CDN. Shared across users, purgeable, geographically close. This is where most of the win is.
3. Your application. Redis, Valkey, or an in-process map. For computed results and database rows.
4. The database. Its own page cache, which you influence but do not control.

Different tools, one idea: keep the answer near whoever asks next.

Two headers do most of the work

Cache-Control states the policy. Learn four values:

• Hashed static assets: app.4f3a9c.js, whose name changes when the content does:

Cache-Control: public, max-age=31536000, immutable A year, and immutable tells the browser not to even revalidate on refresh.

• HTML that everyone sees the same:

Cache-Control: public, max-age=0, s-maxage=60, stale-while-revalidate=600 Browsers revalidate every time; the CDN (s-maxage) serves a copy for a minute and may serve a stale copy for ten more while it fetches a fresh one behind the scenes. That last clause is the difference between a slow page after expiry and a fast one.

• Anything personalised:

Cache-Control: private, no-store

• Something that may be cached but must be checked:

Cache-Control: no-cache, which, confusingly, does not mean “do not cache”. It means “cache, but revalidate before reuse”. no-store is the one that means do not keep it.

ETag is a fingerprint. The browser sends it back as If-None-Match; if it still matches you return 304 Not Modified with no body. You still pay the round trip, but not the bytes. Useful for large responses that rarely change.

The bug that leaks one user’s data to another

This is the dangerous one and it is easy to write.

A page renders “Welcome back, Priya” and is served with Cache-Control: public, max-age=300. The CDN stores it. The next visitor is greeted as Priya, and if the page contains her order history, they can read it.

Three rules that prevent it:

• Never mark a response public if it varies by session. If the request had a session cookie and the response differs because of it, it is private, no-store.
• The cache key must include everything the response varies on. If a page differs by language, the key needs the language: either in the URL or via a Vary header. Vary: Cookie is technically correct and practically useless, because every cookie value becomes a separate entry and your hit rate goes to zero. Put the varying thing in the path instead: /en/pricing, /hi/pricing.
• Watch for Set-Cookie on a cacheable response. A cached response carrying somebody’s session cookie hands their session to the next visitor. Most CDNs refuse to cache responses with Set-Cookie for exactly this reason: do not defeat that default.

Invalidation: change the name, do not purge

Purging is eventually consistent, rate-limited, and awkward at scale. The alternative is to make the URL contain a content hash, so a new version is a new URL and the old one stops being requested. This is what every bundler does with app.4f3a9c.js, and it is why static assets can be cached for a year without risk.

For data caches, the same trick works: put a version number in the key. user:8412:v3. Bumping the version invalidates everything without deleting anything.

The thundering herd

A popular key expires. In the same millisecond, 10,000 requests miss, and all 10,000 go to the database to compute the same value. The database falls over. Then it recovers, the key expires again, and it happens again.

Two fixes, both small:

• Single-flight. Only the first request computes; the rest wait for its result. Most cache libraries have this; it is a lock around the recompute.
• Jitter the TTL. Instead of 300 seconds, use 300 plus a random 0-60. Now the expiries spread out instead of aligning. One line, and it prevents the synchronised version of this problem entirely.

The free path

Cloudflare’s free plan caches static assets globally and will cache HTML if you tell it to: that alone is a CDN most projects never outgrow. nginx’s proxy_cache and Varnish are free and run on your own box. Redis and Valkey are free. Every one of these is a one-afternoon addition, and any of them will do more for your latency and your bill than a bigger server.