TemplateDocs
Incident postmortem
A blameless postmortem skeleton: timeline in facts, impact in numbers, causes in systems — and actions with owners so it doesn't happen twice.
Last updated
Fill in the blanks
Assembled in your browser — nothing you type is stored or sent anywhere.
Your document
# Postmortem: Checkout outage during the Sept 2 deploy
**Date:** 2026-09-02 · **Severity:** SEV-2 · **Author:** Platform team
## Impact
Checkout returned 500s for 41 minutes (14:07–14:48). ~380 carts affected; 62 support tickets; no data loss.
## Timeline (all times IST)
14:03 — Deploy 2026-09-02.3 starts rolling out
14:07 — Error rate alert fires; on-call paged
14:21 — Rollback initiated
14:48 — Error rate back to baseline
## Root cause
The deploy included a migration that renamed a column the checkout service still read. Staging didn't catch it because its replica lags schema changes; the gap is the missing contract test between the two services.
## What went well / what didn't
Well: alert fired within 4 minutes; rollback path worked first try.
Badly: the on-call runbook pointed at a dashboard that was renamed in July.
## Action items
- Platform — add contract test for checkout's read schema — Sep 12
- SRE — fix runbook dashboard links, add link-check CI — Sep 9
*Blameless by policy: this document names systems and gaps, never people at fault. Anyone listed appears only as an actor in the timeline.*
How to use it well
- →Write the timeline first, from logs and chat scrollback, while memory is fresh — every other section is derived from it.
- →The root-cause test: if your sentence names a person, keep asking "what allowed that?" until it names a system, a test, or a process. That's where the fix lives.
- →Cap action items at what will genuinely get done. Three shipped fixes beat eleven ticketed intentions — stale postmortem actions teach the team the document is theatre.