khuynh22/agent-dev-team / skills/release-and-rollback
release-and-rollback
VerifiedRelease and Rollback
Plan the rollback before the rollout. A change that cannot be undone or observed is not ready, whatever its tests say.
Process
-
Answer the rollback question first. What single action undoes this, how long does it take, and can someone do it at 3am from a phone? If there is no answer, stop here and escalate; that is a design problem, not a release problem.
-
Sequence so each step is independently reversible. The safe shape for a data change is four deploys, not one:
- Schema change that is backward compatible (add, do not rename or drop).
- Code that writes both shapes and reads the old one.
- Code that reads the new shape.
- Removal of the old shape, after usage is measured at zero.
Four reversible deploys beat one irreversible one every time.
-
Choose the rollout shape and the abort criterion together. Canary, percentage, ring, or flag. The abort criterion is a number measured from a named query, decided before the rollout starts. Deciding it in the moment, while looking at a graph you want to look good, does not work.
-
Make it observable before it ships. You must be able to answer from telemetry alone: is the new path being taken, is it succeeding, how long does it take. Write the three queries down. See
references/observability-checklist.md. -
Feature flags with a lifecycle. A flag has an owner, a default, a removal date, and someone who can flip it without a deploy. A flag with no removal plan becomes permanent configuration and doubles the state space forever.
-
Write the runbook for the alert, not for the service. The reader is woken up and has five minutes.
-
Check the pipeline. Reproducible build, pinned toolchain, no secret in scope for untrusted pull-request code, and a green run you can point at.
Release plan
## Release: <change>
- **Rollback:** <single action> — takes <duration> — executed in staging: yes/no
- **Steps:** <ordered, each independently reversible>
- **Flag:** <name, default, who can flip it, removal date>
- **Rollout:** <canary %, soak duration, then next stage>
- **Abort if:** <metric crosses <threshold>, from query `<q>`>
- **Observability:** <the three queries: taken / succeeding / duration>
- **Runbook:** <path>
- **Blast radius if wrong:** <who, how many, for how long>
Runbook
# Alert: <name>
**Means:** <the user-visible symptom>
**Check first:** <one dashboard or query>
**Common causes:** <ranked, each with its distinguishing signal>
**Mitigation:** <the action that stops the bleeding, before diagnosis>
**Escalate to:** <who, and when>
Migrations
- Run it forward against a copy of realistic production data, not a fixture.
- Execute the rollback at least once, somewhere real. An untested rollback is a plan, not a capability.
- Long-running migrations run in batches with progress and a resume point.
- Check locking on large tables. "It only adds a column" is true right up until it takes a lock and the site stops.
Verification
- Rollback is a single documented action with a known duration, executed at least once.
- The abort criterion is a number from a named query.
- The three observability questions have concrete queries written down.
- Every alert added has a runbook and an owner.
- Every flag has a removal date.
Red flags
| Thought | Reality |
|---|---|
| "We can roll forward if it breaks" | Roll-forward under pressure turns a small outage into a long one. |
| "The migration is safe, it only adds a column" | With a default, on a large table, with a lock. Check. |
| "We will watch it after deploy" | Watch what? Name the query before you ship. |
| "It passed CI so it is ready" | CI proves the code. It says nothing about rollback or observability. |
| "Alert on CPU" | Alert on the symptom a user feels. CPU is a dashboard. |
| "We will remove the flag later" | Give it a date now, or it is permanent. |
| "Rollback should work" | Should is not evidence. Run it. |
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 197k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |