moonlight-lupin/agent-skills / plugins/dev-pipeline/skills/review-lessons
review-lessons
VerifiedLibrary of lessons from past build-review loops — not a stage. Use when a review or fix round hits a specific case: a reviewer BLOCKING that contradicts the brief, a test that passes but checks nothing, a feature dead in production, SQLite/WAL migrations, MCP tool surfaces, REST API authz, app ↔ external-source sync, Android/APK verification, fill-template extraction, CLI/git mechanics, reviewer outages. Load the one file the case needs; never load the whole library.
references/pitfalls-fix-round-verification.md
View on GitHub# Pitfalls — Fix-Round Verification
Reached from SKILL.md ## Pitfalls. Fix-round and verification semantics: wrong-layer fixes,
red-test triage, gate-function probes, two-sided and live verification. Re-read when
adjudicating a fix round. Split from SKILL.md pitfalls in the 2026-10-16 refactor.
## Contents
- A fix at the wrong layer tests green and does nothing
- Builder live-smoke must never write to production systems
- Self-authored red tests need a two-bug triage: fix import/coverage bugs before dispatch, but RED-assertion failures are the dispatch contract
- Gate functions hide correct behavior behind early returns
- RED tests must fail against the UNFIXED code, not just fail now
- A patch to one half of a bidirectional contract must be probed on the OTHER half
1. **A fix at the wrong layer tests green and does nothing — trace the payload from transport to validator** — when a fix guards a payload shape (e.g. an error raised when `data_preview` replaces `data` in a provider response), the unit test that calls the validator DIRECTLY with the new shape can pass while production never reaches that code: an earlier function in the call chain (e.g. `_execute_composio_tool` returning only `response.data` and discarding the sibling field) strips the shape before the validator runs. The test then validates dead code. Fix rule: when hardening a validator for a shape change, first trace where that shape is BORN (transport/API boundary) and what it looks like at the validator's entry — then write the hardening at the layer that still sees the full envelope, and add an end-to-end test that mocks the COMPLETE transport envelope through the public method. Caught by Codex R1 on a connector field-briefing cycle (Aug 2026): the data_preview offload error was unreachable despite a passing unit test.
2. **Builder live-smoke must never write to production systems** — briefs that include a live verification step must explicitly wall off the WRITE path: "do NOT run sync-out/sends/publishes against the live sheet/system — the orchestrator does that." Verify the READ path live (fetch, list, count rows), mock the write path in tests, and reserve the first production write for the orchestrator's own verified run. A builder exercising a write path under test conditions corrupts production data.
3. **Self-authored red tests need a two-bug triage: fix import/coverage bugs before dispatch, but RED-assertion failures are the dispatch contract** — when the red contract tests fail at baseline, classify each failure first. A test that imports one name and calls another (or asserts coverage of an API the spec never named) is an ORCHESTRATOR test bug: fix it directly (test-only code, see `review-findings-adjudication.md` — own the fixture fix), because leaving it teaches the builder the wrong surface and invites a compensating hack (a builtins binding shipped a hidden global into every importer until caught). But a failure whose assertion encodes the intended fix (red because the shim still exists, red because the artifact still gets written) is the DELIVERABLE — commit it red, name it in the brief as "currently RED for this reason; your job is to make it green by deleting X", and verify the same assertion flips green after the round. The trap: a builder blocked between an attested old assertion and a new fix requirement will add a detection shim to satisfy both — the re-attestation rule below is the escape hatch, and the red-first check is how you detect it (the re-attested test must fail BEFORE dispatch, not pass).
4. **Gate functions hide correct behavior behind early returns — rebuild the probe from the producer's shape, not the brief's** — a `capture_snapshot`-style gate returns None when the payload lacks gate keys (a `summary` dict) or nests sections differently than assumed; a probe from a guessed minimal payload then reports "snapshot stored: False" on correct code and triggers a false fix round. Before flagging a gate as broken, read its early-return conditions and re-derive the minimal VALID payload from the real producer's return statement (grep the collector's literal dict construction), not from the spec prose. (Subsumes the former duplicate entry 'Probe gate-style functions with a payload that satisfies their acceptance rule', which added: the same false-failure reproduces with `_extract_counters`-style gate functions, and the probe must be re-derived from the producer's return shape, not from the brief.) Related: same-day snapshots may REPLACE rather than append (a daily-kind snapshot captured twice in one UTC day overwrites the first) — a probe capturing two same-day snapshots then expecting a two-point series reads an empty series that is correct behavior, not a bug. Read the replace/prune rules before interpreting series results.
5. **RED tests must fail against the UNFIXED code, not just fail now — prove each failure names the defect** — a RED contract test can pass accidentally against the broken implementation (weak assertion) or fail for a test bug, and a green run over vacuous tests certifies nothing. Before dispatching the fix round: run the red tests against the CURRENT (unfixed) module, and for each failure read the assertion that fired — it must point at the defect the finding describes. Vacuous tests are worse than missing ones because they certify the fix and block nobody: reviewers prove them vacuous by running them against the unfixed module and getting a pass. Known vacuous shapes: a concurrency test that never runs concurrent writers; a monkeypatch that fires before the module's first checkpoint (the mutation lands inside the legitimately-read baseline); an outcome test whose assertion the WRONG code path also satisfies ("fresh text appears" — the conflict path produces it too). Strengthen weak assertions to pin the mechanism: byte-equality across cycles (not just size), absence of the wrong code path's artifacts (no conflict wrapper, no backup), refusal-vs-merged status, and a source-level ordering contract for lifecycle code (e.g. assert unlink precedes LOCK_UN in the release path).
6. **A patch to one half of a bidirectional contract must be probed on the OTHER half** — fixing the acquire side of a lock does not fix the release side; both reviewers reproduced the two-holder race with the acquire fix in place because the release path still unlinked after LOCK_UN. When a finding names a lifecycle (acquire/release, begin/commit, read/write), patch the named half but probe BOTH halves before declaring the blocker resolved — the decisive probe is the one crossing the boundary between the halves (a waiter acquiring in the unlock→unlink gap).
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |