Pattern: watch something, say when it changes

The shape

Check a source on a schedule or subscribe to its events. Compare what you see with what you saw last time. Decide whether the difference is worth telling anyone. Tell them.

Price changes, competitor pages, regulatory notices, a supplier’s stock levels, a slow-moving dataset, a folder that should receive a file every morning. The pattern is everywhere and it is simple in a way that hides two hard parts: deciding what counts as a change, and not becoming noise.

Diff structured state, never the raw page

Fetch the same web page twice and the bytes differ: session identifiers, timestamps, rotating advertisements, a “3 people are viewing this” counter. Diff the HTML and everything is a change, every time.

Extract the fields you actually care about, store those, and compare those:

state = {"price_minor": 129900, "in_stock": True, "sku": "AX-4"}
if state != last_state:
    notify(diff(last_state, state))

Hash the extracted state to detect change cheaply, but store the fields themselves, because a notification saying “something changed” makes the reader do the work you were supposed to do. Send the before and after values in the message.

The model’s job here is small

Detection must be deterministic. Never ask a model “has this changed”: it will answer differently on identical inputs, and you will have built a monitor that fires at random.

The model is useful after detection: summarising what a fifteen-clause change to a policy document actually means, or judging whether a change is material enough to wake somebody. Deterministic detection, model-assisted interpretation. Keep the boundary sharp, because a stochastic detector produces the worst failure available to this pattern, which is a signal nobody can trust.

Noise is the failure mode

An alert nobody reads is worse than no alert, because it trains people to ignore the channel, including on the day it matters. Alert fatigue is the whole reason monitors get switched off.

Three controls:

Hysteresis. Do not alert every time a value crosses a threshold. Require the condition to hold for three consecutive checks, and require a return to normal before you will alert again. This removes flapping, which is most of the volume.

Batching. Anything non-urgent goes into a digest at eight in the morning: one message, everything that changed overnight, sorted. Reserve immediate notification for things somebody would act on at 2am, and be honest about how short that list is.

Actionability. Every alert states what changed, what it was before, why it might matter, and links straight to the thing. If the reader has to open three tabs to understand it, they will not, twice.

Watch the watcher

This is the part that gets missed and it is the most important paragraph in the lesson.

A monitor that has crashed sends no alerts. A world in which nothing is changing also produces no alerts. From the reader’s side these are identical, and the silence is reassuring in both cases, so a dead monitor can go unnoticed for months and be discovered on the day it was needed.

The fix is a heartbeat, sometimes called a dead man’s switch. Every successful run pings something; if the ping does not arrive on schedule, that raises an alert. Healthchecks.io has a free tier and self-hosts; or write a last_run_at timestamp somewhere and have a second, separate job complain when it goes stale. The two jobs must not share a machine or a scheduler, or a single failure silences both.

Alert on the absence of work, not only on the presence of problems. This is the single most transferable idea in the operations half of this course.

The free path

cron with curl, jq and a JSON file of last state does this properly in about twenty lines. changedetection.io self-hosts and handles page watching with a UI, including rendering JavaScript-heavy pages. RSS still exists on more sites than people assume and is a subscription rather than a poll, which is cheaper and kinder to the source.

Whatever you use, respect the source: a sensible interval, a real user agent with contact details, and attention to robots.txt. A monitor polling every thirty seconds is indistinguishable from an attack from the other end, and getting your address blocked is the least bad outcome of that.