TemplateDocs

Incident postmortem

A blameless postmortem skeleton: timeline in facts, impact in numbers, causes in systems — and actions with owners so it doesn't happen twice.

Last updated

Fill in the blanks

Assembled in your browser — nothing you type is stored or sent anywhere.

Your document

# Postmortem: Checkout outage during the Sept 2 deploy **Date:** 2026-09-02 · **Severity:** SEV-2 · **Author:** Platform team ## Impact Checkout returned 500s for 41 minutes (14:07–14:48). ~380 carts affected; 62 support tickets; no data loss. ## Timeline (all times IST) 14:03 — Deploy 2026-09-02.3 starts rolling out 14:07 — Error rate alert fires; on-call paged 14:21 — Rollback initiated 14:48 — Error rate back to baseline ## Root cause The deploy included a migration that renamed a column the checkout service still read. Staging didn't catch it because its replica lags schema changes; the gap is the missing contract test between the two services. ## What went well / what didn't Well: alert fired within 4 minutes; rollback path worked first try. Badly: the on-call runbook pointed at a dashboard that was renamed in July. ## Action items - Platform — add contract test for checkout's read schema — Sep 12 - SRE — fix runbook dashboard links, add link-check CI — Sep 9 *Blameless by policy: this document names systems and gaps, never people at fault. Anyone listed appears only as an actor in the timeline.*

How to use it well

  • →Write the timeline first, from logs and chat scrollback, while memory is fresh — every other section is derived from it.
  • →The root-cause test: if your sentence names a person, keep asking "what allowed that?" until it names a system, a test, or a process. That's where the fix lives.
  • →Cap action items at what will genuinely get done. Three shipped fixes beat eleven ticketed intentions — stale postmortem actions teach the team the document is theatre.

Part by part

## Timeline (all times {{tz}})
One stated timezone. An incident spanning teams produces timestamps from several, and reconciling them afterwards is both tedious and error-prone.
## Root cause
Singular in the heading and rarely singular in reality. It is the section that most needs the discipline of asking what made the failure possible rather than what triggered it.
## What went well / what didn't
Both directions. Recording what worked is what stops a postmortem removing a control that was actually doing its job.
Blameless by policy
The policy stated in the document itself, not just in the meeting. People who expect blame withhold detail, and the withheld detail is usually the part that prevents a recurrence.
never people at fault
Makes the standard explicit and checkable. A postmortem naming a person is a document that will make the next one less honest.

Common mistakes

  • Naming who made the mistake.

    It ends the flow of information. The next person who notices something wrong hesitates, and the hesitation costs more than any individual error the postmortem could have addressed.

  • A single root cause.

    Real incidents have several contributing factors, and picking one produces a fix that leaves the others in place. Ask what made it possible, not just what set it off.

  • Action items with no owner or date.

    Unowned actions do not happen, and a postmortem whose actions were never done is how the same incident recurs with a document explaining it in advance.

Questions

Why blameless specifically?

Because the alternative reliably produces worse information. Systems that punish error get fewer reports of error, not fewer errors — and the reports are what let you fix anything.

How soon should it be written?

Within a few days, while the timeline is still recoverable from memory and chat logs. Later, the reconstruction becomes a story people agree on rather than a record of what happened.

Who should read it?

Anyone who could hit the same class of problem, which is usually wider than the team involved. A postmortem kept within one team teaches one team.

What if the cause was a person's mistake?

Then the question is what allowed a single mistake to cause an incident. A system where one wrong command takes production down has a design problem, and the person is the last line rather than the cause.

Related in Templates