The day you run two copies

The second instance breaks things the first one hid

One process is a simple world. Anything you keep in a variable is available to every request, a file written by one request is readable by the next, and a scheduled task runs once because there is one of it.

Start a second copy and every one of those assumptions stops being true. The site does not fall over; it misbehaves in ways that look like bugs in your code. This lesson is the list of what breaks, because knowing it in advance is the difference between an afternoon and a fortnight.

The five things that break, in the order you meet them

1. Sessions in memory. A user logs in on instance A and their next request lands on instance B, which has never heard of them. The symptom is being randomly logged out, roughly half the time with two instances. Fix: put sessions in Redis, in the database, or in a signed cookie. Not sticky sessions, for reasons below.

2. Rate limits and counters in memory. With two instances your limit of 100 requests an hour is now 200, and neither process knows about the other. Anything that counts across requests needs shared storage.

3. Uploads on the local disk. The file is written to instance A, and the request that displays it hits instance B, which returns 404. This one is covered properly in the next lesson; the short version is that local disk is scratch space, never storage.

4. Caches in memory. Not wrong, but now inconsistent: two processes hold two versions of the same value and invalidating one leaves the other. Acceptable for short TTLs, dangerous for anything a user just edited.

5. Scheduled jobs. Your nightly email runs twice, because both instances have the same cron. Every scheduled task needs a lock: a row in the database taken with a conditional update, or a single dedicated scheduler process.

The pattern under all five: state that lives inside a process cannot be shared by processes.

Sticky sessions are a trap that works

A load balancer can pin each user to one instance, which makes problems 1, 2 and 4 disappear without changing any code. It is tempting, and it is why so many systems have it.

What you have bought: when that instance restarts, and it restarts on every deploy, everybody pinned to it loses their state at once. Load also stops distributing evenly, because a long-lived user stays where they were regardless of how busy that box is, and autoscaling cannot drain an instance without dropping people.

Use stickiness deliberately for things that cannot move, such as an open websocket. Do not use it as a substitute for externalising state.

Statelessness is a property you can test

The test is blunt and takes two minutes: while using the site as a logged-in user, restart the process. If your session, your draft, your cart or your upload disappears, that state was in the process.

Run the same test with two instances behind a proxy on your laptop. Docker Compose with --scale web=2 and any small proxy in front is enough. Almost every problem in this lesson shows up locally in ten minutes, and none of them show up in a single-process development server, which is why they are usually discovered in production.

Autoscaling does not do what people expect

Three honest limitations:

• It is slow. A new instance takes 30 seconds to several minutes to boot, install and warm up. A traffic spike that arrives in ten seconds is over before capacity appears. Autoscaling handles daily curves well and sudden spikes badly.
• It scales the thing you measured, not the bottleneck. Scaling on CPU when the constraint is database connections adds instances that queue harder on the same database, and, past that point, makes latency worse rather than better.
• Scale to zero means cold starts. Free tiers that sleep after inactivity give the first visitor a multi-second wait, and that visitor is often the one you cared about.

A fixed number of instances with headroom is a legitimate, cheaper answer for most small services. Measure the ceiling as in the previous lesson, run at half of it, and revisit quarterly.

Where the real limit lives

Adding web instances is easy because they are stateless once you have done the work above. What does not scale by copying is the thing they all share: one database. That is the subject of the next lesson, and it is why “just add servers” stops working at a predictable and surprisingly early point.

The free path

Docker Compose with two replicas and Caddy or nginx in front costs nothing and reproduces every problem here. Redis or Postgres serve perfectly well as your shared session and counter store: you almost certainly already run one of them, and adding a second dependency for this is rarely justified at small scale.