Knowledge

What a postmortem is

Collect the lessons of an incident systematicallyand turn them into team knowledge

The point of a postmortem is not to assign blame. It is to learn from the incident and work out what to improve. Maximising what you learn from a failure — and sharing it — is what raises reliability across the organisation.

A fixed format lowers the cost of writing one and leaves something readable behind. The example below is the minimum that works.

What goes in the report

Item

What to write

Example: demand-forecasting app, overnight job failure (SEV2)

Summary

What happened, in one sentence

The overnight retraining job failed, so the next morning's forecast never appeared on screen

Impact

Whose work was affected, and how

The business team used the previous day's forecast for morning planning. No work was halted

Timeline

When it started, was noticed and was fixed — and what was done

02:14 job fails → 08:30 a user reports it → 09:10 cause identified → 10:05 rerun and recovered

Cause

What triggered it, and why it was not caught

An automatic library update changed the input format and the load failed. Nothing alerted on the failure, so it went unnoticed until morning

Preventive actions

What will be done, by whom, by when

1. Pin the version (ALBY / 30 Sep) 2. Alert automatically on failure (ALBY / 30 Sep) 3. Add a check to the update procedure (client and ALBY / 15 Oct)

Making it work in practice

When to run one

Always for SEV1 and SEV2. For SEV3 and below, run one when the same thing keeps happening, or when service came back without anyone understanding why.

Following through on the actions

A postmortem is only worth as much as the preventive actions that actually get done. Review them regularly — in the weekly team meeting and the monthly all-hands.

The culture it needs

  • Psychological safety, so people can describe what really happened
  • Treating postmortems as valuable work rather than paperwork
  • A standing place to look back, not a one-off

References

Talk to us about building and running your cloud infrastructure