Moving data between systems
Reversible where possible, rehearsed against a copy, and always producing a record of what changed.
The most expensive incident available in this problem space is not an outage. It is a data change nobody can undo, run against live records, that quietly altered more of them than intended.
Every such change is either reversible or explicitly documented as not reversible, and the second category requires a reason. Reversibility is not free — it constrains what the change may do — and the trade is made deliberately, not discovered at the moment it matters.
Every change is rehearsed against a restored copy of live data before it touches anything live. Test fixtures do not contain the row that has been null since 2019, and that row is the one that fails.
The work reports what it changed: how many records, of which kinds, and what the totals were before and after. A change that reports nothing has told nobody whether it worked, and the absence of an error is not evidence that the right thing happened.
Backfills are batched and resumable rather than run as one long transaction. A backfill that fails at eighty percent should resume at eighty percent, not roll back and start over, and a long-held lock on a busy table is an outage by another name.
Changes to structure are separated from changes to content, and both are separated from the release that depends on them. Adding a column, populating it, and beginning to read from it are three steps that can each be verified, rather than one step that either works or produces a difficult evening.
Where a change alters the meaning of existing data rather than its shape, the prior values are preserved rather than overwritten. A definition that changed is a fact about the history, and a system that erases it makes every comparison across that boundary silently wrong.
This work runs at a time chosen for the business instead of for convenience, with someone available to watch. The riskiest moment in a system's life is not the release itself; it is the twenty minutes afterward when a problem is still cheap to reverse and nobody is looking.
Nothing runs against production without a current, verified backup — verified meaning restored, not merely present.
What this does not cover.
- Irreversible changes run without a verified restore of the backup taken beforehand.
- Destructive structural changes released in the same step as the software that depends on them.
- Silent migrations that report no counts.
Software Engineering
Specification before code
The document that names every entity, every state, and every failure path — and why a project that cannot produce it should not start.
Types as guarantees
Making the likeliest mistake in a system impossible to express, rather than merely against the rules.
Internal tools
Authenticated back-office systems for the work currently living in a spreadsheet.
Commerce and payments
Checkout, registration, and subscription flows built against the client's own payment account.
Interfaces between systems
The interfaces a system exposes, and the contracts it holds itself to once something depends on them.
Testing
Coverage aimed at what would actually be expensive to get wrong, instead of at a percentage.