“It should always work” is not a target
Ask anybody what reliability they want and the answer is 100%. It is the wrong answer, and not for the reason people expect.
100% is unreachable, because your users reach you through networks, DNS resolvers, browsers and phones you do not control, and those fail more often than your service will. Long before your own uptime becomes the limiting factor, you are spending money to improve a number nobody can perceive. Every extra nine costs roughly ten times the previous one, and buys a difference the user cannot distinguish from their own wifi.
So the useful question is not “how reliable can we be” but “how reliable do we need to be, and what are we willing to give up for it”. That number, written down, is a service level objective.
The three words, in order
- SLI: the indicator. A measurement. “The proportion of requests that return a non-5xx status within 500 ms.”
- SLO: the objective. A target for that measurement over a window. “99.5% of requests, over 30 days.”
- SLA: the agreement. The same idea in a contract, with money attached when you miss it. Most small projects have no SLA and should not pretend otherwise.
Write the SLI as good events divided by valid events. That formulation forces you to define both halves, and the arguments happen at the right moment: before the incident, not during it. Are health checks valid events? Are requests from a bot? Is a 429 to an abusive client a failure? Decide once, write it down.
What the numbers mean in minutes
Over 30 days, allowable downtime is:
- 99%: 7 hours 18 minutes.
- 99.5%: 3 hours 39 minutes.
- 99.9%: 43 minutes.
- 99.95%: 21 minutes.
- 99.99%: 4 minutes 20 seconds.
Read that last line and notice what it demands: nobody can be woken, diagnose, and fix anything in four minutes. Four nines is not an operations target, it is an architecture: automatic failover, no single points of failure, and a team on rotation. A one-person project promising four nines is promising something it structurally cannot deliver.
For most small sites, 99.5% or 99.9% is honest, and the first is easier to defend than people think.
The error budget is the useful half
If the objective is 99.9%, then 0.1% of requests are allowed to fail. That is not a shameful residue; it is a budget, and it is meant to be spent.
The budget converts reliability from an argument into arithmetic. Plenty of budget left this month means ship the risky change, run the migration, try the new provider. Budget exhausted means stop shipping features and spend the time on reliability instead. The rule is written down in advance, so the decision during a bad month is not a negotiation between the person who wants to ship and the person who was woken up.
A small team can run this informally: one number on the dashboard, checked at the start of each week.
Burn rate, which is how you alert on it
Alerting whenever the error rate is non-zero produces noise. Alerting when the budget is fully spent is too late. Burn rate sits between: it asks how fast you are consuming the budget relative to the whole window.
A burn rate of 1 spends exactly the month’s budget in a month. A burn rate of 14.4 spends it in about two days, and a common pairing is: page if you have burnt 2% of the budget in one hour, ticket if you have burnt 10% in six hours. The fast alert catches outages, the slow alert catches a steady drizzle that would otherwise never trip a threshold.
The practical benefit is that one alert covers both the sudden total outage and the slow bleed, and neither needs a hand-tuned latency threshold.
Measure it where the user is
An SLI computed from your own server logs cannot see the requests that never arrived: DNS failure, TLS expiry, a load balancer returning 502 on its own, the whole region unreachable. Those are exactly the worst outages.
So compute the indicator as close to the user as you can afford: the CDN or load balancer’s logs rather than the application’s, plus an external synthetic check from somewhere else entirely. Free tiers of uptime services do the second part; a cron job on a different provider’s free instance does it for nothing.
The version for one person
You do not need a platform for this. You need:
- One SLI, defined in a sentence, in the repository.
- One target, with a window.
- One number on a dashboard: budget remaining this month.
- One rule about what happens when it runs out.
That is a working reliability practice. Everything else (multi-window multi-burn-rate alerting, per-endpoint objectives, formal reviews) is scaling that up, and it is worth adding only when the simple version has been used for a few months and found insufficient.
Every lesson here is open. Your progress is recorded once you pass the Agents and Automation exam.
Check your understanding
A service with a 99.9% monthly availability objective has burnt 60% of its error budget by day four, entirely in one 25-minute incident. Under a budget-based policy, what does that imply?
Pick the one you would defend. Your answer stays private.
Stuck on something?Ask Tri, your AI tutor