Releasing changes
The commit that was built is verified by its identifier, and the release is verified against the live address instead of against a build status.
A green build and a live site are different claims, and treating them as one is the most common release incident there is. Not a failed build — those announce themselves. A successful build of the wrong commit, confirmed by looking at a dashboard that reports what the platform did instead of at the address a customer would actually visit.
Releases are automated from version control. The platform watches one line of work, builds what is on it, and releases that. Nobody uploads a finished build from a laptop, because an upload has no recorded change behind it: what is live can drift away from what was reviewed without anything registering that it has, and every incident afterwards begins with a question nobody can answer.
That question is the specific one worth designing for — what is live right now. An automated release answers it with a recorded identifier that can be checked against what was intended. An upload answers it with somebody's recollection of a Tuesday.
The identifier is checked, not assumed. After a release, what the platform reports as live is checked against what was intended, because a build can succeed against stale material, an unmerged line of work, or two changes that arrived out of order, and all three produce a green status over the wrong version.
Release is then verified against the live URL. A marker present only in the new build — a version string, a hash, a value that changed — is polled for at the public address until it appears, which converts "the release finished" into "the change is being served". Those are separated by layers of caching and occasionally by a routing rule still pointing at the previous version.
A verified release includes re-checking the pages that matter afterwards, in production, over the public address. Not a check against a developer's own machine and not a screenshot from a build log — the actual site, on the actual domain, as a stranger would reach it.
A health check gates the release instead of decorating it. The platform must have a route it can call to establish that the new process is not merely running but able to serve — because a process that boots successfully and then fails on its first real request will otherwise receive traffic, and the release will be reported as a success while the site returns errors. The check has to point at a route that genuinely exists; aimed at a path nobody implemented, it fails every release instead, which is the same mistake pointing the other way.
Changes are reviewed at a real address before they are merged, at a temporary address of its own instead of on somebody's laptop. A screenshot proves the change rendered once on one machine; a URL lets the person who asked for it look at the thing itself, which reliably surfaces the misunderstanding that a screenshot conceals.
The checks run as part of the release itself and not only on a developer's machine. A check that exists locally is a check that runs when somebody remembers to run it — the same failure the gates were written to eliminate, reintroduced one level up. If the gates pass locally and are absent from the pipeline, the pipeline is not enforcing anything.
Rollback is established before it is needed, and the mechanism is the same one used to release: promote a version already known to work. A rollback path that involves a person editing something by hand under pressure is not a rollback path.
The criterion for whether to release is not the day of the week. The useful questions are whether the change can be reverted quickly and whether somebody who understands it will be reachable for the next hour — and where the answer to both is yes, Friday afternoon is unremarkable. Where the answer to either is no, Tuesday morning is not safe either, and the folk rule has been standing in for the real one.
Configuration values are part of the release rather than adjacent to it. The most reliable way to produce a build that works everywhere except production is to add a variable in one environment and not the other, and it fails while running rather than while being built, which means it fails in front of somebody.
Structural changes to stored data run before the software that depends on them, as part of the release itself instead of as a step somebody remembers. Where such a change cannot be applied without downtime, that is stated in advance and scheduled, rather than discovered when the release takes the site down.
Destructive structural changes are never automated against data that matters. An operation that will silently discard information to make the structure match is not something to place in a path that runs on every change — the blast radius of that arrangement is any structural edit, ever, and the damage is not recoverable from the application side.
Releases are small and frequent rather than large and occasional. A release containing one change has one suspect when it misbehaves; a release containing thirty has thirty, and the time spent identifying which is usually longer than the time saved by batching them.
The build is reproducible from version control alone. If producing a working release requires a file somebody has locally, a step nobody wrote down, or a service configured by hand two years ago, then the system depends on a person, and that will be discovered at the worst available moment.
Build configuration is held in version control as a reviewed file, not only in a hosting dashboard. Console settings are unversioned, unreviewable, invisible in a diff, and absent the moment the service is recreated — which is exactly when nobody remembers what they were.
Releases are recorded: what went out, when, from which version, and by whom. Most of the value arrives during an incident, when the question is what changed in the last day, and the alternative is a group of people trying to reconstruct it from memory while the thing is still broken.
None of the above is sophisticated, and that is rather the point. Every item is a small piece of discipline that costs a few minutes at release time and removes an entire category of afternoon spent establishing what is actually running. The systems that decay are not the ones with bad architecture; they are the ones where nobody can say with confidence what is deployed.
What this does not cover.
- Uploading a build directly to a live service. Every release has a recorded change behind it.
- Destructive structural changes automated against real data.
- Treating a successful build as evidence that the change is being served.
Managed Services & Infrastructure
Backup and recovery
A backup nobody has restored is a belief; recovery time is measured by performing it, not estimated from the size of the file.
Environments
A non-production environment earns its cost by being wrong in the same ways production is.
Monitoring
Watching the things a business would actually notice losing, rather than the things that are easy to graph.
Keeping software current
Few enough third-party components that keeping them current stays possible three years after launch.
Ownership and access
The domain, the payment account and the cloud accounts are the client's; handover is a transfer of access, not a negotiation.
Coverage and response
Stated hours, not implied availability, because a commitment is only worth what somebody can plan around.