phuryn/product-engineering / ai-shipping/skills/release-gate
release-gate
VerifiedBefore 'push it', 'ship it', a deploy, release or go-live, or calling a stage done: the gate, the owner's preview, go/no-go, promote, prove it serves, roll back.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 26.8 KB |
| receipts.md | markdown | 9.1 KB |
| references/checklist.md | markdown | 22.0 KB |
| references/go-live.md | markdown | 11.5 KB |
| references/netlify.md | markdown | 11.2 KB |
| references/runbook.md | markdown | 17.2 KB |
references/runbook.md
View on GitHub# Any release as one logged checklist
How to run a release that is more than one web deploy — a package, an editor extension, desktop installers, a store listing, a client and the backend it talks to — as one ordered list with evidence at every step. Learned on a remote-control product: a relay service with a web client on one side, and on the other an editor extension, with desktop installers, that drives a CLI agent, published to two extension registries. The web-app promotion (`checklist.md`) is one instance of this shape; here is the general one.
## One list, run with a log
- **The procedure is one ordered list in one place.** Each step names its command, what counts as a pass, and the evidence it leaves — a SHA, an exit code, a report path. Reasons live elsewhere and are followed when a step surprises you, not instead of the step. (`2026-09-27`, the remote-control product: the procedure had been spread over five files and was being run from memory. A month earlier, two patch releases went out without the independent review, because the file that required it had silently stopped loading and nothing announced its absence.)
- **Run it with a log.** Copy the step numbers into a `Release run — <version>` section of that release's plan (or the backlog when there is no plan) and tick each one with its evidence. **A skipped step is written down as skipped, with the reason.** Close the log with the versions, the SHAs, the evidence paths and every skip.
- **When a step proves wrong or missing, fix the list in the same pass.** That is what keeps it current.
- **If a script exists for a step, use the script.** A script encodes the procedure; an agent who reinvents a step gets it subtly wrong. (`2026-07-27`, a hand-rolled package step shipped an extension that would not load. `2026-08-16`, a hand-rolled release attached a package built locally for testing — pointed at the staging backend by design — to the public release, where it sat for about four minutes before anyone noticed.) A guard that makes the wrong artifact unbuildable is the backstop, not the rule.
- **A script the release calls refuses what it does not read.** Each command accepts only the options it reads — its own list, not one shared by every command — and refuses the rest before doing anything; a dry run is a command of its own, never a flag a command might ignore. A configuration that names no target is refused, never defaulted to production. A refusal names a bare flag and never echoes its value, and a status command reads a not-ready answer (a `503` with a body) rather than failing on it, because not ready is when it is needed. (`2026-10-24`, a monitoring product's operator CLI: `deploy --dry-run` deployed development's worker for real, because the option was ignored; one allow-list shared by every command let `--project-ref <another project>` through to a command that deployed its own checkout's environment anyway; an unknown `--name=value` was echoed in the refusal, so a pasted connection string would have printed its password; and the edge's status command said "command failed" exactly when the edge was not ready.)
- **Who decides.** The owner decides *that* a release happens — explicitly; an authorization is spent by the release it authorizes — and any version beyond a patch. Everything else in the list is procedure and runs without asking. **Never infer a go** from an ambiguous reply ("ok", a thumbs-up, silence); when it is unclear, ask, even for "it's only the website".
- **The version is a patch unless the owner said otherwise.** The test for a minor is whether users gain something they could not do before, not how large or visible the diff is: spending a minor on a refinement makes the next real one say less. (`2026-08-24`, a minor was spent on a running count added to an existing indicator.)
## A. The candidate is finished
1. **Trees clean and pushed, read rather than assumed.** Read `git status --short` and `git ls-files --others --exclude-standard` before staging; never `git add -A` blind. Anything machine-specific or agent-local that is not ignored (local tool permissions, an agent's scratch output) goes into `.gitignore` in the same commit — in a public repository a stray file is published, not just committed. A generated or vendored copy is compared with its source by hash, never by eye.
2. **Live checks at the candidate commit**, against the real model or the real CLI the product drives. They cost money, so they sit outside the gate (`test-harness`):
- **Name the cheap models up front.** A cost guard that refuses an unnamed model makes every capability read *not exercised* — the guard working, not the product failing. (Rediscovered at three releases in a row, `2026-10-13` to `2026-10-18`.)
- **The pass is the exit code plus a person's reading.** Every unhandled entry in the report is explained, and every render report the check writes is read and judged. "The model did not do the thing" is a re-run of that scenario, never a pass.
- **A finding triaged as pre-existing is accepted inside the check's own table**, marked as an accepted gap with its backlog entry, as well as in the backlog. It stays visible in every report and stops failing runs; only a new unknown fails. An un-accepted known finding blocks the release at its gate.
- **When the version of the vendor tool the product drives changed since the last release, run the full live suite, not the partial one.** (`2026-09-28`, the remote-control product: a partial run was clean after triage, and the full run inside the release script then stopped on an event kind only a later scenario emitted — a whole release cycle lost.)
3. **The full-range independent review**, every repository the release touches, ranges computed from git off the last release (`independent-review`). **A round that ran out of turns and reported nothing is a failed round, not a clean one**: give the wrapper a turn cap that fits the subject. (`2026-09-28`, a two-commit fix review exhausted a 60-turn default and exited 1 having reported nothing.)
- **Before hardening anything a round calls a security boundary, write one sentence: if this is wrong, who can do what, and the damage is what.** When the actor is the user themselves — their own account, device or data — it is a scoping or presentation rule, not a boundary: build the cheapest thing that is right in the steady state. A rule that partly applies itself (an idle timer, a heuristic, a derived timestamp) cannot be a boundary, because access would change with nobody acting. Repeated findings in one mechanism are evidence about the requirement, not the code. And compare where the complexity lands against what it prevents: machinery in the path that decides whether the core action runs costs more than a cosmetic outcome is worth. (`2026-08-10`, the remote-control product: "archived projects should not be reachable remotely" was treated as a permission against the owner's own signed-in device. Seven implementations and six rounds produced thirteen findings, seven High, each introduced by the previous fix, with machinery in the session lifecycle. The owner asked whether this optimised for 0.1% of users; the replacement — a filter on what a remote is shown — removed 331 lines.) The rest of triage is `independent-review` § Triage.
4. **Pins, by kind.** A version pin that is only a comparison point ("an update is available") takes the version the live checks ran against. A pin that selects what an installer downloads is a change of its own, with its installer tests, never a release-day bump. Record both in the log.
## B. Docs and bookkeeping — one pass, then one review
5. **Docs-staleness, every release-facing document**: the README, the store listing, the agent instructions, the architecture doc, the test doc, the package manifest's description and metadata. **When the release changes a state every doc describes** (production exists, a service moved, a capability went live), grep for the phrases of the old state and fix them in one pass, not line by line as reviewers name them (`checklist.md § 1`, with its receipt).
- **A generated listing is regenerated, never hand-edited**, and a test holds the committed file equal to the generator's output, so a hand-edit fails. (An extension's store README is generated from its repository README.)
- **Re-read the public marketing copy against what ships**, before every production promotion: feature work keeps moving what the site promises, and on a pricing page a drifted promise is worse than a support ticket. (`2026-07-30`.)
- **Then one independent docs round, with one question: which sentences assert behaviour the code does not have?** Name the range, say it is documentation only, exclude prose you would have written differently and omissions that do not mislead. A document is a promise; a confidently wrong sentence is worse than a missing one. (`2026-09-11`, the remote-control product: the round found two false claims in user-facing copy, both past the author's own read and a green suite.) **A rewrite inherits the claims it does not examine** — one of the two had been restated, in new words, from the sentence it replaced.
6. **Changelog section and version**, terse.
7. **Backlog sweep: close every entry the release resolved.** Strike the heading with the version it shipped in, say what shipped and where it lives, **keep the original diagnosis** underneath, and never delete the entry. **Check its premises, not only its status**: a stale premise survives the fix and sends the next reader to plan against a world that no longer exists. Better still, close the entry in the commit that fixes it; the sweep is the backstop. (`2026-08-22`, three entries found stale in one morning, one of them a fix that came within a brief of being re-implemented.) **An entry whose revisit trigger this release fires is acted on, or re-set with a new trigger and the reason** — a trigger left firing is a deferral nobody decided (`checklist.md § 1`, with its receipt).
## C. The backend first, whenever the client needs anything new from it
- **A client build that relies on new backend behaviour is not published until the backend carrying it is in production** — not merely on the main branch or a development deploy. A backend that predates a new message drops it silently: no error anywhere, the data simply never arrives.
- **The other direction is a contract: every backend release stays usable against the last published client.** Protocol changes are additive; the client gates a new affordance on having received what feeds it (capability detection), never on a version number, and degrades rather than rendering a control the other side cannot answer. Do not bump a protocol version to protect an ordering you control: **order the release, don't break the contract.** The harness that holds it is in `test-harness` (`references/harnesses.md`, a frozen-fixture compatibility suite).
- **The order:** backend gate → development deploy → smoke → production promotion → every backend runtime the push does not build brought current, a fleet one host at a time (`checklist.md § 4`) → production smoke → client release tests → client tag and publication. Record both repositories' commits with any cross-repository result.
- **Promote by fast-forward, never through a pull-request merge.** A merge commit on the production branch breaks the fast-forward invariant, and the next promotion can no longer fast-forward.
- **An unauthenticated smoke cannot see the authenticated mile.** After a deploy that touches sign-in, sessions, ownership, entitlements or connection admission, run the authenticated canary too, with dedicated canary credentials (`test-harness`).
- **A development environment on a different identity instance is not evidence about production's sign-in.** Say so rather than implying that green on development means green on production.
## D. The client release
- **A leak check on the outgoing range, when a public repository must not carry something** — a private sibling's name, commercial plans, a surface not yet announced. Run it on a ref (`git grep -inE "<terms>" <ref>`) or on `--cached`, never on a bare `git diff`, which cannot see an untracked file. **The pattern is a backstop; read what you are about to push** — the rule is not a word list. (`2026-08-11`, a new untracked fixture carrying a private repository's name passed a pre-staging diff check and sat in public history until the next commit's check caught it. `2026-08-22`, the pattern of the day passed a commit naming an unannounced surface in merchandising language.)
- **Run the suites CI does not run before the release script, not inside it.** Found that way, a broken harness costs a harness fix; found mid-release, it costs a half-run release (`test-harness`, a new precondition).
- **The release script** runs the gate (integration and live suites included), packages, commits, pushes, **waits for CI on the pushed SHA before tagging** — without that, a release can be tagged and published before a second CI job goes red — tags, and **attaches the artifact this run just produced, never a leftover.**
- **A release with several artifacts is finished only when every one is attached.** Read the release's asset list; a dispatched build that "read like the job was done" is not evidence. (`2026-08-09`, a release shipped with only the extension package attached, because the first of two installer dispatches had run.) Build release artifacts **from the tag**, so they are exactly the released code; test the branch-built artifacts **before** tagging — the tag build is not a substitute for testing. A build matrix whose release step needs every leg attaches all or nothing, so a download page never offers two versions at once.
- **Publishing is not availability.** A registry may scan a new version before listing it, and a store may take minutes to show it. (As observed on 2026-09-02: the VS Code Marketplace took about ten minutes to show a publish, and its `lastUpdated` did not move meanwhile, so an agent reported a successful publish as failed; Open VSX scans new versions before listing them.) **Poll for the version; never infer from a timestamp**, and check that the query itself does not exclude a version still validating. **Never announce a listing as live**; say when it should appear.
## E. After the stores
- **Deferred deletions only once the store serves the new version.** A published listing can load images by absolute URL from the repository's main branch, so deleting or renaming one breaks the live store page after the release, with nothing local to notice. (`2026-08-07`, a release left alt text and a broken-image icon on a registry's listing because only the repository README was updated.) A test fails on a dangling image reference in every listing source.
- **When the product updates installations already in the field, getting the release onto them is part of the release — and verified by reading the installation, never the rollout tool's summary.** "Scheduled" is not "done". Grep the installed bundle for a string the release *deleted*: its absence proves the new code is installed, which a version file does not (the same probe as `SKILL.md § Verify`). (`2026-09-03`, every installation reported armed and every one still ran the previous build hours later, the owner's included, which kept reproducing a fixed bug.)
- **A machine set up by hand is not evidence about the image you ship.** When a release adds something the shipped image or installer must contain, the release changes the image; development machines where it was installed by hand pass every test. (`2026-09-25`, a newly supported agent reached the desktop and never the cloud image.)
- **Issues: sweep every open one**, not only those the changelog names, and read each thread before crediting a fix to it. For each the release fixed: a signed comment saying what shipped and in which version, a label asking the reporter to confirm, and the issue left **open** for the reporter to close. **An issue is done only once the fix is installable from the store the reporter uses**: a confirmation label that starts an auto-close clock while the store still serves the previous version can close an issue nobody could verify. Never `Fixes #N`, which closes it on merge.
- **A public comment describes nothing unreleased in the present tense.** Committed is not done: say "will be in <version>", in the future tense. (`2026-09-25`, a comment said a defect "is fixed" while the fix was committed and unreleased; the reporter tested the release they had, still saw it, and from their side it was the third claimed fix that was not there.)
- **A reopened issue means the fix addressed a different cause.** Confirm a diagnosis against the reporter's configuration, not against a reproduction you built. (`2026-09-16`, a fix for a setting that "snapped back" was right for a cause the reporter did not have.)
- **A contributor stays visible as a contributor.** Merge so their commit survives with their identity; adjust with commits on top; if history must be collapsed, carry `Co-authored-by:` with the address on their commits; never re-implement a pull request as your own change, however much of it you rewrite. **When merging a contributor branch, read the diff — never take `--theirs` wholesale on shared docs.** (One merge reverted three sections of the test and architecture docs, each fix clobbered again by the next conflict resolution.)
- **Close the log.**
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |