The first day of a security incident

It does not look like the films

A security incident usually starts as something small and confusing. A user says their account posted something they did not write. A cloud bill has an unfamiliar region on it. A support agent notices two accounts sharing one email. A researcher emails you a screenshot.

The outage runbook from earlier in this course does not fit, because the priorities are different in one specific way: you must preserve evidence while you stop the damage, and the instinct to “clean it up” destroys the information you will need for every decision afterwards.

The first hour

1. Write down the time and what you saw. Start a document. Timestamp everything from here.
2. Preserve before you change. Snapshot the machine, export the relevant logs to somewhere the incident cannot reach, and take a copy of the affected database. Do not rebuild the server yet. Once logs roll over, often in days, the answer to “how did they get in” is gone permanently.
3. Contain, narrowly first. Revoke the compromised credential, disable the affected account, block the specific access path. Wholesale shutdown is available and expensive; use it when narrow containment is not working.
4. Rotate credentials that could plausibly be exposed, in the order the earlier rotation lesson gave: new key, deploy, watch, disable.
5. Get a second person involved. Not for expertise; for the decisions. Alone at 2am is when people delete things they needed.

What not to do: log into the compromised machine and start poking, which contaminates timestamps; announce a cause in the first hour; or promise users that nothing was taken before you know.

Establish scope with evidence, not with hope

The question everybody wants answered is “what did they get”. Answer it from logs, not from reasoning about what should have been possible.

• Which credential was used, and what could that credential reach?
• What queries or requests were made with it, and how much data left?
• When did it start? The gap between the first sign and your detection is the number that matters most, and it is usually longer than people expect.
• Is the access still live? Assume yes until you have positive evidence otherwise.

If you cannot establish scope, say so. “We do not yet know” is a legitimate finding and a far better public statement than a confident claim you later withdraw.

The obligations, in shape rather than in detail

This is where the honest answer is that the rules differ by jurisdiction and by the kind of data, and this course cannot tell you what applies to you.

The shape, roughly, in several regimes: if personal data is exposed, a regulator may have to be told within a short window: 72 hours from becoming aware is the well-known figure under the European regime, and other countries have their own periods, some shorter for particular sectors. Affected individuals often have to be told directly when the risk to them is high. Some sectors (health, finance, children’s data) carry stricter duties. Contracts with business customers frequently impose faster notification than the law does, and people forget to read those.

What follows for you, practically:

• Find out before an incident which regimes you are under and what the clock is. Write it in the runbook with the actual deadline.
• Keep a record of when you became aware, because the clock starts there and you will be asked.
• If personal data of real people may be exposed, take advice from somebody qualified in your jurisdiction. Nothing here is legal advice, and this is one of the few places in this course where the stakes justify professional help.

Telling people, without making it worse

When you notify, say: what happened, what data was involved, what it means for the reader, what you have done, what they should do, and when you will update again. Plain language, no minimising verbs, no “sophisticated attack” unless it was.

The reputational damage in published breach cases is rarely from the breach itself. It comes from the delay, the initial denial, or the phrase that turns out to be untrue. A short, early, accurate notice that says “we do not yet know the full scope” ages far better than a confident one that ages badly.

If passwords were involved, force a reset and end all sessions: the session lesson’s revocation design is what makes that possible in minutes rather than days.

Not every incident is a breach

Keep the categories separate, because they need different responses and different words:

• Credential stuffing: attackers using passwords leaked from another service. Your system was not breached; individual accounts were. Say that precisely, help the affected users, and add the defences from the sessions lesson.
• A vulnerability report with no evidence of exploitation. Fix it, thank the reporter, and check your logs for use before deciding whether anybody needs to be told.
• An internal mistake: a public bucket, an over-broad query in an export. Still a data exposure, still possibly notifiable, and the containment steps are the same.

Afterwards

Run the postmortem as usual, with two additions: the detection gap in hours, and an honest account of what evidence you did not have and now will. Most security post-incident actions are unglamorous: turn on logging you did not have, remove a credential that should not have existed, restrict a permission that was broader than the job needed. Those are the ones that make the next one smaller.