Incident Response Checklist for Eng Teams

Detect & declare

0/5

Confirm the alert or report reflects a real, current problem before declaring an incidentChasing a stale or flapping alert wastes the exact response capacity a real incident will need.
Assign an incident commander as the first action, even before root cause is knownWithout one clear owner, multiple people investigate in parallel and duplicate or contradict each other's work.
Set an initial severity level based on user impact, not on how the bug looks in codeSeverity should track what users are experiencing right now, which is often disconnected from how gnarly the cause is.
Open a dedicated channel or thread for the incident, separate from general chatA single thread of record stops critical context from getting lost across DMs and side conversations.
Start a timeline immediately, timestamping what's observed as it's observedReconstructing a timeline from memory after the fact loses exactly the details a good retro depends on.

0/4

Look first for a fast mitigation — rollback, feature flag, failover — before hunting for root causeRestoring service comes before understanding why it broke; root-causing can happen once users aren't affected.
Check recent deploys and config changes before assuming a novel bugMost incidents trace to something that changed recently, and that's the fastest thing to check and revert.
Confirm the mitigation actually restored service, using real user-facing signals, not just that it was appliedA rollback that doesn't fix the symptom, or fixes the wrong one, needs to be caught immediately, not later.
Watch for a second failure mode the mitigation itself might introduceA rushed rollback or failover under pressure can trade one incident for a smaller, different one.

0/4

Post an initial status update within the first few minutes of declaring, even with limited informationA short 'we're aware and investigating' update prevents stakeholders from filling the silence with worse assumptions.
Update status at a predictable interval, not just when there's meaningful newsRegular cadence itself is reassuring — silence between updates reads as nobody working on it even when someone is.
Keep customer-facing and internal updates factual and free of speculation about causeA guessed-at cause that turns out wrong has to be walked back publicly, which costs more trust than the delay would have.
Notify the specific stakeholders who need to act (support, leadership, affected customers), not just a general channelA broadcast update that never reaches the people who need to act is functionally a missed notification.

0/4

Confirm the underlying cause is fixed, not just the immediate symptomA mitigation that only masks the symptom leaves the same incident free to recur, often at a worse time.
Verify monitoring and alerting would catch a recurrence of this specific failureIf the incident was possible before existing alerts caught it, closing that gap is part of actually resolving it.
Get explicit agreement from the incident commander before declaring the incident closedA single clear close prevents the ambiguity of half the team standing down while the rest keeps investigating.
Send a final update confirming resolution to everyone who received earlier status updatesPeople notified of a problem should also be explicitly told it's over — silence after the last update leaves it open.

0/4

Hold a blameless post-incident review while details are still fresh, ideally within a few daysWaiting weeks loses the specific details that make a review useful, and lets people fill gaps with hindsight bias.
Write down the actual timeline, including detection delay and time to mitigate, not just the fixDetection and response time are usually the most improvable parts of an incident, and they're invisible without a timeline.
Turn findings into specific, owned, dated action items, not a general 'be more careful'Action items with no owner or deadline are the single biggest reason the same incident repeats.
Share a summary broadly enough that the same failure class gets recognized faster next timeThe value of a review compounds when other teams learn the pattern before they hit it themselves.