ccoalm/ccl-skills / skills/skill-extraction-workflow
skill-extraction-workflow
Flagged复盘 / 沉淀 / 总结经验 / 补进技能 / 技能缺陷 / 流程优化 / would other teammates hit this / will we forget next time / teamwide recurrence / extract skill / post-mortem / 深度 review·审计技能仓库 / 对标外部技能包找 gap(benchmark superpowers·gstack 等) / 核查全局安装点(~/.config/opencode 等)旧快照是否遮蔽本仓技能 / 本仓(ccl-skills 等共享技能仓)OpenCode 项目配置·命令治理 → turn reusable lessons into durable CCL skills.
references/validation-and-landing.md
View on GitHub# Validation And Landing
Use this before claiming a skill extraction is complete.
## Static Validation
- Parse all skill frontmatter as YAML. Frontmatter validators that parse the `description` field MUST use a YAML parser (e.g. Ruby `YAML.safe_load(text, aliases: false)`), not a single-line regex. Block-scalar descriptions (`description: |` followed by indented multi-line content) silently collapse to the literal `"|"` under naive regex parsing, which makes length, empty, and content checks blind. The parser's exception handling must rescue `Psych::Exception` (or the equivalent parent class), not only `Psych::SyntaxError` — when `aliases: false` is set, alias YAML raises `Psych::BadAlias`, which is a different subclass.
- When a skill's frontmatter `description` changes, grep the skill body for terms in the new description and flag contradictions before landing. Common failure mode: description claims the skill covers X, body has a stage / scope line saying "for X, route to other-skill". The reverse also fails: description Skip-when routes Y elsewhere, body Use-this-skill section claims Y. Resolve by editing whichever side is wrong; do not commit a contradiction.
- When SKILL.md or a reference declares a "gate", "rule", or "fail-closed" enforcement, verify the implementing script actually enforces it. Common failure mode: SKILL.md says "audit fails closed when alias YAML missing", script only runs the audit if an environment variable is set and silently exits 0 otherwise. If the script cannot enforce the policy, soften the SKILL.md language to describe the actual mechanism (e.g. "maintainer MUST run X before commit; script verifies only when ALIAS_AUDIT_CMD is exported") rather than overpromising. Doc-vs-script reconciliation is part of every commit that touches policy text or enforcement code.
- Check that every skill-internal referenced file, especially `references/*.md`, exists and is under the skill directory.
- Scan changed skill files for source repo names, business terms, personal paths, credentials, and deprecated tool names.
- Confirm execution references can be used without access to internal source artifacts unless the task is explicitly about auditing, updating, or re-extracting from those artifacts.
- Confirm provenance-heavy details are isolated in source maps or evidence maps, and that normal routing does not force users to read raw source inventories.
- Check that the skill does not contain long verbatim source dumps, transcript dumps, blockquote-heavy copied text, or unnecessary raw logs.
- Check that source links used as evidence are specific inspected artifacts, not generic homepages, search pages, topic pages, or placeholders.
- Check that verification output, install visibility, and review status are real. If a gate was not run, record it as missing or pending.
- Capture a verification command's exit status directly — never through a pipe stage without `pipefail` (`cmd | tail` makes `$?` the filter's status; separate the run from the filtering, enable `pipefail` and save `$?` immediately, or snapshot `PIPESTATUS` in the very next statement — any intervening command destroys it). Know the macOS bash 3.2 trap hazard (empirically isolated): with `set -e` set, an **unbound-variable expansion abort** (`set -u`) enters the `EXIT` trap with `$? = 0` — so a successful cleanup trap (or a status-re-raising one, `trap 'rc=$?; …; exit $rc' EXIT`) returns 0 and cannot recover that class (a trap that itself exits nonzero imposes its own unrelated status instead); explicit `exit 1` / failed-command statuses ARE preserved by a re-raising trap. So: use re-raising cleanup traps that disable `errexit` inside the handler before exiting with the saved status (protects the explicit-failure class), never take "exit 0 + trap present" as proof against expansion aborts on bash 3.2 — check the output for abort messages, or run the suite on bash ≥ 4 (CI's Linux bash is unaffected).
- Parse/read the COMPLETE validator output, not just its final line. `check-ccl-skills.sh` exits 0 even when the private R0 alias audit never ran. When `ALIAS_AUDIT_CMD` is unset the validator runs the deterministic public fallback (`generic-r0-leak-scan.sh`) over added-diff markdown lines and prints `generic_r0_leak_scan_ok` plus `alias_audit_unavailable`, `r0_status=public-fallback`, and a final `ccl_skill_check_interim_ok` token; that fallback fails closed on high-signal leaks but does NOT prove the private alias audit ran. (Only when the private audit ran clean does the final token become `ccl_skill_check_clean_ok` with `r0_status=private-ok`; a deprecated `ccl_skill_check_ok` line is still printed for old consumers but is no longer the last line or a sufficient signal.) Any `alias_audit_unavailable` / `ccl_skill_check_interim_ok` in the output MUST be recorded as **private R0 not run / missing clean-landing R0 evidence** — neither the deprecated `ccl_skill_check_ok`, the interim token, nor a clean `generic_r0_leak_scan_ok` softens or overrides it. Record R0 as satisfied only when the output shows `alias_audit_ok`, the record cites a named private-profile result, or an explicit risk-owner waiver exists; otherwise the change may proceed only as an `interim` commit/branch/Draft MR with `R0 pending maintainer audit`, not as landing-clean or merge-ready (see `r0-leakage-audit.md`).
- Check the extraction charter. Purpose, scope, depth, root cause, evidence plan, and completion standard must be visible in the durable trace for any extraction beyond wording-only cleanup.
- Check RCA depth for any extraction beyond wording-only cleanup (Deep RCA five moves, scaled to the failure's size). For any non-trivial failure the RCA must **widen** to multiple contributing factors (not a single chain), **counterfactually test** each to rank causal weight — **without dropping redundant safeguards / secondary controls**; the counterfactual ranks levers, it does not prune defence-in-depth — and end at **mechanical control(s)** on the failure class (a source row, routing rule, execution recipe, validator, test, review gate, provenance rule, or final-response constraint) plus the feedback that confirms it fires, passing the firing-path proof. A file-level or broader extraction must also separate active vs latent conditions and record local rationality. Small single-rule non-wording clarifications scale down to a widen-check plus the control and do not need the full multi-factor table — but failure shape, not diff size, sets depth: a one-line patch for a recurring, under-trigger, or multi-cause failure still uses the full method, and do not relabel a semantic/routing edit as "wording-only" to escape it (strict wording-only test). A retro fails this check if it bottoms out at one linear chain, at "agent careless" / "should have known" (hindsight), or drops a failed safeguard as "not necessary".
- Check claim-to-evidence parity. Source maps, final summaries, and commit messages must not say "full", "complete", "all", "re-read", or "inventory" unless the changed files include evidence at that depth.
- Check that no rule was landed through post-hoc evidence fitting. Require a ledger origin for each landed source-derived rule. A landed rule needs `observed` origin, source IDs read before drafting, and concrete observations. If a rule began as `hypothesis`, confirm it was converted with evidence, narrowed, routed, discarded, or left out before landing.
- Check provenance-to-executable landing. If a source register, source map, evidence map, or target-output map says a mechanism was extracted into a target skill/reference, the target file must contain a usable rule, recipe, checklist, or acceptance gate for normal users. A provenance-only note does not count as a landed skill update.
- Record the provenance-to-target diff check for any extraction beyond wording-only cleanup: source mechanism, provenance row or source id, target file, executable landing line or section, test/acceptance owner, and status. Missing target lines must be fixed before completion or the claim must be downgraded.
- Treat `pending` provenance-to-target rows as blockers for complete/final wording. Treat `downgraded` rows as acceptable only when the source map and final response also downgrade the mechanism to provenance-only, routed, or insufficiently evidenced.
- Check coverage labels are honest and match the charter depth: wording cleanup/no new source read, targeted check, file-level refresh, node/artifact-level inventory, full workflow extraction, or generator/tooling change.
- Check source-read failure recovery. If any source read timed out, failed transport, returned partial output, or was too large, the validation record must show a smaller or different read attempt, recovered evidence, remaining gap, and whether failed reads were excluded from coverage claims.
- Check correction handling. If the change follows a user/reviewer correction about extraction quality, require an RCA entry: previous wrong behavior, contributing cause(s), missed evidence or gate, corrective rule, target, and verification.
- Check objective-closure parity for every extraction or retrospective over a behavior-changing delivery. Read the delivery's population preflight (`product-rd-workflow/references/implementation-completeness-and-minimality.md` §Objective Closure): a missing line, an unquoted or unresolvable `does-not-fire`, or a single-instance exception resting on an implementer-authored or same-delivery source counts as `fires`. The extraction must put the objective's outcome classes and the accepted coverage side by side whether or not the preflight fires and whether or not a prior mismatch report exists; a parity claim without that enumeration is a gap. **Parity is judged against the product-side outcome-class table's own columns, not a looser summary** — per class: scope (`in`/`out`/`deferred`/`absent`) with its locator or mechanism, the member scope it holds over, the implementation surface, the fresh evidence over that member scope, and the status — all six columns, since dropping the evidence column lets `in / all members / handler X / satisfied` pass with no command, trace, or query behind it — plus each class's independently-failable variant inventory, because a single satisfied `negative` row that never enumerates timeout, denial, conflict, and dead-letter is incomplete on the product side and parity must see that. A retrospective row saying "negative → covered, recovery → covered" carries none of those columns and is itself the gap; "no visibly smaller coverage" is not a pass when the columns were never filled. A fired preflight additionally requires the member-population comparison — the same enumerated-vs-covered reconciliation the delivery gate owes, never a stricter one invented here, and an open population or a per-operation bound that defers a member additionally requires the continuation comparison. When a batch, sample, subset, slice, quota, or partial coverage is smaller than the objective — or either required comparison is missing — the target-output map must include `product-rd-workflow` as well as this workflow, and the exposure is a path-(i) failure class (proxy/subset/slice/local-bound narrowing), so a prevention point lands here too. **A missing delivery-side record is not discharged by the extraction alone**: when the delivery carries no population or outcome-class record, the extraction must also name the delivery-side repair — the acceptance record reconstructed, or the delivery explicitly reopened or marked `interim` — because citing the now-existing rule while the examined delivery still has no objective-closure record leaves the product gate un-fired.
- Before editing for such an exposure, walk an objective-closure inventory: the required positive, negative, corrected/recovered, and ambiguous-or-superficially-successful outcome classes; each local bound's effect on the total obligation; and the continuation/finality condition that prevents permanent omission or starvation. Name the exact §Objective Closure sub-clause that fires and quote the applied acceptance text. Confirming that the rule text exists in the product owner is only a precondition — validation passes only when the mapping records the quoted sub-clause plus the product-owner edit or the existing-rule reference it relies on. A phrase-only check, a project-only batching fix, an omitted-count report, a chat explanation, or a bare "already covered" claim does not close the exposure. Keep the section name synchronized if the rule is renamed.
- Check task-retrospective handling. If the user asked to summarize this task, summarize lessons learned, or review what went wrong, require an RCA-backed classification for each durable lesson: skill, validator/tooling, project artifact, memory, or final response only.
- Check `agents/openai.yaml` or equivalent UI metadata still matches the skill's trigger and purpose.
- Check every changed skill makes four discovery and routing facts explicit: core responsibility, trigger or when-to-call, expected output artifact, and boundaries versus sibling skills.
- Check the source coverage matrix for the requested extraction boundary. Every relevant source category should be used, routed, discarded, or marked unavailable with a reason.
- Check execution readiness: the skill should tell a future agent how to analyze the task, make ownership decisions, debug failures, test behavior, and verify the real output surface when those steps are relevant.
- Check split boundaries: if two surfaces require different commands, tools, runtimes, release gates, or verification evidence, confirm this was evaluated as split evidence. Require separate skills or explicitly routed subflows only when trigger, owner, and workflow are also clearly different.
- Check stack-specific sibling evidence: when a changed skill is tied to a language, framework, client runtime, or testing stack, the validation record must show the sibling-generalization mini-map decisions (`update`, `unchanged`, or `route-to-shared`) and the final diff must match those decisions. Static YAML/reference validation is not a substitute for this behavioral/routing evidence.
- **Checklists earn their length by anchoring to a failure class, not by being thorough.** Every checklist item in a skill or reference should map to a concrete past miss / regression / review finding (or to a plausible future failure named in the RCA) — items that exist only because "it would be good to remember" become noise that future readers skim past. Prune items that no failure class anchors to; do not add items without one.
- Run `git diff --check`.
Prefer the bundled validator when available. From the ccl-skills repo root:
```sh
./skill-extraction-workflow/scripts/validate-skill.sh /path/to/skill
```
Use the repository root to validate all immediate skills:
```sh
./skill-extraction-workflow/scripts/validate-skill.sh /path/to/ccl-skills
```
When the skill is installed outside the source repo, resolve the installed `skill-extraction-workflow` directory and call its `scripts/validate-skill.sh` with the target skill root or skill repository root as the argument.
If the validator reports `missing_required_command`, keep the failure visible and complete the static validation bullets manually.
## Read-modify-write example code
Relocated verbatim from `SKILL.md`'s `Validation & the dual-track gate` Core Rules group (the entrypoint keeps a one-bullet summary that points here); it applies to every reference file or skill example that ships runnable code against external mutable state, and wording changes here go through the same shared-skill gates as an entrypoint edit.
- Reference example code that performs read-modify-write on external mutable state (Bitable records, database rows, file contents, API state) must:
- Read failure: raise explicitly; never return `{}`, `""`, `None`, or any empty-success value that silently drops the prior state.
- Write failure: raise or skip, never silently continue.
- Uniqueness invariant: when the example declares, assumes, or depends on one, detect violations such as duplicate unique keys and raise before propagating bad state.
- Lost-update control: use optimistic concurrency controls (ETag, version field, CAS, transaction, or compare-and-swap), append-only API semantics, or an explicitly declared single-writer precondition — RMW examples that silently assume no concurrent writers will produce lost-update bugs under normal conditions.
- Data-loss anti-pattern: log-and-continue after a read failure on an append-only field.
## Behavioral Validation
- For new skills or major workflow changes, use `writing-skills` for RED-baseline/test-first methodology — **and the firing point is BEFORE drafting the body, not only before finalizing**. Eval-first authoring for a NEW skill (or a new hard-rule section): (1) write the evaluation scenarios first — **at least three** for a new skill (both the vendor's published authoring guide and the high-star practice pack converge on three-plus scenarios before body text; a single-rule edit may scope down to that rule's own scenario); (2) run them WITHOUT the skill and record the observed failures verbatim — and for a discipline-slip failure (the agent knows the rule and skips it under pressure), capture the agent's rationalizations word-for-word: each verbatim excuse is the raw material for one rationalization-vs-reality row and one red-flag line in the skill text (the discipline-slip form in `rule-consolidation.md`'s form-by-failure table); an invented hypothetical excuse does not qualify — counter only what a run actually said, and don't add rows for excuses no run produced; **a no-skill control that does not exhibit the failure is a stop signal — do not author guidance for a failure you cannot observe** (record the null finding instead; this is the pre-draft face of "Evidence must come before new rules"); (3) draft the **minimal** content that addresses the observed failures, then re-run the same scenarios WITH the skill; (4) when a later run, review round, or live miss surfaces a NEW rationalization for an existing discipline gate, add its explicit counter row to that gate's table and re-run the tempting scenario — counter tables accrete from observed excuses across rounds, never from imagination. The code-level RED-GREEN-REFACTOR method (write the failing case first, watch a fresh agent violate the rule WITHOUT the skill, then add the skill and watch it comply) is owned by `superpowers:writing-skills` + `superpowers:test-driven-development` — **if installed, route there; otherwise apply the RED-baseline rule inline** (manually record the without-change failure and the with-change compliance). This is the skill-authoring face of **eval-driven development** (for a behavior/routing change, run the scenario before you finalize; never special-case the scenario just to make it pass) — borrow the *principle*, not a claim of production-grade eval rigor.
- **What makes a `RED-baseline` valid is executed-and-recorded vs narrated — not recorded vs live.** Any evidence form (before-after diff, golden trace, or pressure scenario) is valid when it actually records the without-change failure AND the with-change compliance with a locator + expected-vs-actual (see `dual-track-review-gate.md`). Prose that merely *describes* an expected failure without running it is not a baseline; a pressure scenario you actually executed and recorded is.
- **A claimed impossibility is falsified in-env before it lands (authoring and reviewing alike; relocated from `SKILL.md`).** Deferring work behind a conservative-sounding stub ("not implemented", "needs a future tested wrapper") is the avoidance form of a blocked-verification claim: it *feels* safe but ships an unverified impossibility as durable behavior — distinct from a real `unavailable`/`pending`-with-remediation+residual-risk record, which is what remains after a safe attempt was genuinely impossible (the attempt itself stays within the safety boundaries the `SKILL.md` rule names). Failure shape: a fallback reviewer lane shipped as a permanent fail-closed stub on an unverified "permission probe not implemented" premise that a ~60-second live tool run refuted, re-enabling the lane.
- **The falsifying operation has exactly two admissible forms, and a search hit is neither (detail for the `SKILL.md` landed-conclusion rule).** (1) **Executed path** — run the suspected mechanism on the path that actually failed and collect an observation that only THIS cause predicts. Naming the call site is the entry bar, not the operation: **reachability alone rules out non-execution and nothing else** — a wrapper can demonstrably run while a downstream stall is what caused the failure, so "I watched the suspected code execute" clears no cause. The probe must produce a discriminating consequence (the mechanism's own signature in the output, a value only it would set, a timing or state it alone explains); if the run cannot separate the suspect from the alternatives still on the table, it has not falsified anything and form (2) is required. (2) **Single-variable control** — a paired probe whose arms differ in exactly ONE variable. Enumerate EVERY precondition of the predicate under test (each threshold, each uniqueness or vocabulary requirement, each surrounding-state assumption) and confirm both arms equal on all but the varied one; an arm differing in two preconditions yields a verdict attributable to neither, and it fails silently because the verdict still reads as decisive. A shared surface here is the source-register row, the commit or merge-request body, a durable note or memory, and the final response's causal account. The control leg a deferred registration owes (`source-to-skill-extraction.md`) is this rule's narrow instance, and the differential attribution a `RED-baseline` row owes (`dual-track-review-gate.md`) is its encoded form — both assume the arms are otherwise equal, which is the part that has to be enumerated rather than assumed. **For a MEASUREMENT the varied arm is the INSTRUMENT, not the subject, and it qualifies only on three counts together.** (a) **Power** — it could have returned a different reading, whether a number or a qualitative or derived verdict the tool states about the subject: an arm sharing the first's method, assumptions or error mode, or sweeping a parameter the reading does not depend on, could never have disagreed and certifies nothing, however independently built or undisclosed. (b) **Demonstration** — the record names a mover, an actual setting or input on which the arm returned a different value; "it could have disagreed" asserted without one is the same unfalsified claim a level up. (c) **Binding** — that mover moves the SAME property that was reported, at or spanning the configuration that produced the reported reading; a mover for a neighbouring property, or one only reachable far from that configuration, leaves the reported reading untested. Agreement is evidence only where disagreement was possible, demonstrated, and demonstrated here. A value that moves when the tool or its settings move is a reading of the tool, not a property of the subject; an instrument that ran to completion has demonstrated execution, not validity; and a maturity field the output declares about itself ("draft", "uncalibrated") is a hint, never the check.
Failure shape: a gate rejected a ledger anchor; a `force_encoding(BINARY)` call found by grep in the gate script was recorded as the cause and landed in a merged PR body, a durable memory note, and a round charter. The suspected line was never on the checked path — that check resolves its blobs through a different helper — and the real cause was an unbounded string replace in the round's own edit, which had modified a pre-existing ledger row. Three paired probes built to test the theory each differed in more than one precondition (anchor length against the threshold, uniqueness within the file, vocabulary membership), so no verdict was attributable; the first of them produced the wrong cause. Withdrawing it cost corrections on three surfaces, against about a minute for the executed-path question. Measurement shape: an uncalibrated detector borrowed from another corpus printed a coverage fraction; it was reported as a finding about the artifact, defended in a summary, and written into a durable evidence document and a memory note. An independently built measurement of the same quantity contradicted it, and sweeping the detector's single noise parameter moved the original figure across nearly the whole range it could take — the number had been a reading of the tool all along. Withdrawing it cost corrections on three surfaces, against one command for the second measurement.
- **A headless code-writing RED/GREEN needs a *fair* violation-tempting scenario + an *independently-valid* objective measure — a capable agent complies on a clean task.** When the rule governs how an agent WRITES code, a clear well-specified prompt usually makes even the no-rule baseline produce compliant code (zero delta → no RED), so a clean-task baseline proves nothing. Surface the RED with **realistic inherited pressure** — a real legacy/house convention the rule must override, or a genuinely ambiguous spec — **NOT an explicit instruction to emit the anti-pattern**: leading the baseline directly into the violation launders a constructed failure into "RED" and is fabrication (the same defect as the rule above), so record why the prompt is fair and mark any direct-leading scenario synthetic/advisory, not RED. Score both runs with an **objective measure that detects the ANTI-PATTERN** — the rule's own checker counts ONLY if it was independently validated first (held-out positive/negative fixtures + a documented residual boundary, so RED genuinely fails); a same-change checker that merely whitelists the expected GREEN form makes GREEN trivially true. Run the agent **cross-model / fresh-context** so the baseline isn't primed by your session. If even the fair tempting baseline complies, that is an honest finding — the rule's marginal value is in edge/legacy cases, not the common one — record it, don't manufacture a RED.
- **Optional real-agent RED-baseline (F4 Tier-3).** For a routing-surface or hub-skill change you can execute the baseline through the in-repo F4 Tier-3 harness (`scripts/eval-golden-trace.rb`; contract in `eval-routing.md`) instead of a hand-recorded scenario: run it manually **twice** — first on the checkout WITHOUT your change, then WITH it (the harness does **not** auto-checkout or auto-diff; you run both and preserve both reports as the without→with evidence). Caveats that keep it honest: (a) it is **advisory + non-deterministic + small-N** (a few hub traces, structured *routing* assertions only, statuses PASS/FAIL/INCONCLUSIVE) — a without-change run that returns PASS or INCONCLUSIVE does **not** establish a RED; only an actually-observed miss does; (b) prefer a **pre-existing frozen trace** — a trace you author alongside the change is self-authored evidence (the runner checks `frozen_at_sha` ancestry but does **not** detect a trace added/tuned in the same change), so the reviewer must confirm it is not trivially fail-before/pass-after; (c) it stays **optional and never mandated** — a properly executed-and-recorded pressure scenario remains a valid `RED-baseline` for most changes.
- Create at least one pressure scenario: a realistic prompt where the agent should use the skill and avoid a known failure mode.
- A pressure scenario for extraction quality must reopen source evidence during the validation pass. If the task claims UI/UX, behavioral, psychology, implementation, testing, or launch-acceptance extraction, reopen at least one relevant design/source artifact and one runtime/code artifact unless explicitly unavailable or out of scope.
- Record the pressure-test result as one of: `gap found and landed`, `gap found and routed`, `no new gap found`, `blocked/unavailable`, or `static review only`. Only the first three count as behavioral validation.
- For deep or complete extraction requests, validate against the charter instead of only the changed text: did the extraction cover the declared source boundary and root cause at the declared depth?
- For design skills, include a pressure scenario that would otherwise pass broad rules but still look mediocre; verify the skill forces layout/state/visual acceptance decisions.
- For design skills extracted from Figma or frontend code, validate that the skill remains useful without source access and satisfies the executable design checklist in `source-to-skill-extraction.md`.
- For design, UX, frontend/client, product workflow, architecture, or review skills, include a scenario that would tempt the agent to add reasonable but unsourced guidance. Verify the skill forces source refresh, honest coverage labeling, ledger origin checks, or leaves unverified external expertise out of executable guidance.
- For client development skills, include a pressure scenario that requires analysis, debugging, and testing, not only implementation; verify the skill names the runtime surface checked.
- For high-impact workflow skills, also ask an independent reviewer or agent to check trigger conflicts, overfitting, missing gates, and duplication.
- For RCA-driven updates, ask the reviewer to test the exact previous failure mode, not only the new wording.
- For shared or generator-backed skills, prefer a stricter profile: lifecycle test, install-visibility test for each host, doc-contract check for trigger wording, and negative checks for forbidden source leakage.
- For small updates to existing skills with NO change to trigger / scope / routing / validation / acceptance, a focused pressure scenario can serve as the behavioral-evidence row (or `semantic-control` with reviewer confirmation). If any of those change, `RED-baseline` is required. Independent review alone is NOT behavioral evidence for a non-wording shared-skill change — record the behavioral-evidence row separately (see `references/dual-track-review-gate.md`).
- Prefer validation that hides the intended answer; the goal is to see whether the skill guides behavior, not whether the reviewer agrees with a leaked conclusion.
Independent review prompt template:
```text
Review the changed skill without assuming the intended answer is correct. Focus on trigger fit, sibling-skill conflicts, overfitting to source artifacts, business/domain leakage, no-source-access usability, broken reference routing, context load, validation coverage, install visibility, and whether the change would alter future agent behavior. Return findings first with file/line references, then residual risks. Do not edit files.
```
## Cross-File Wording Drift Detection
When a rule lives in two or more authored files — `SKILL.md` entrypoints and `references/` files alike (typical cause: cross-stack mirror — the same operational invariant has a producer-side rule in one stack's skill or stack part and a consumer-side rule in another's; or a test-strategy mirror of a production rule; or a routing rule that lives both in `SKILL.md` and in its owning reference) — fix-one-file drift is a recurring failure mode. A correction to one copy silently leaves the sibling copies stating the old (wrong) rule, and the next reader picks the wrong one. Throughout this section, "authored files" means authored `SKILL.md` entrypoints and `references/` files; generator output and private archives are excluded.
Examples observed in real extraction work:
- "Discovery cache miss → fallback to baseline lane" written in two refs while two other refs (correctly) required fail-closed for canary/stress/tenant routes — four-file P1 contradiction surfaced only by adversarial challenge.
- "Strip inline binary bytes from response" written as production rule in two refs but mirrored as a test-strategy assertion that pushed tests to break the wire contract — fix landed in one ref, drifted in another.
- Timeout layering ordering flipped in one ref and not the mirrors — fix introduced a new direction-inverted statement adjacent to the old one.
### Authority is the invariant, not the newest edit
The corrected wording does not become the truth because it is newest. The authority is the **validated invariant or owning contract** — the architectural rule whose correctness has been independently confirmed (review + challenge passes, external spec, or platform owner sign-off). Sibling reconciliation reconciles every copy to that invariant. Picking "the latest commit" as the source of truth lets a wrong fix propagate across every sibling and erases the contradiction signal that dual-track was supposed to surface in the first place.
### Mirror set is enumerated up front
The mirror index lives in **one owning authored file** — pick the architecture skill that defines the invariant when there is a clear owner, otherwise pick the producer side. That file carries the canonical `Mirrors:` block naming every sibling and what it owns ("producer", "consumer", "test obligation", etc.). **Every other sibling carries a reciprocal pointer back to the owner** — a one-line note such as `Mirror: this rule is owned by <owning-path>; see Mirrors block there.` That way a maintainer who starts from any sibling (the test obligation, the consumer-side reference, a `SKILL.md` entrypoint) finds the index in one hop, instead of having to know which file owns the invariant a priori. The source register and the private provenance archive are NOT acceptable homes for the index — a check that depends on private-archive data has false negatives for any reader who does not load the archive (most readers).
### The discipline
Before commit, for any rule that appears in more than one authored file (`SKILL.md` entrypoints and `references/` files) or is part of a documented cross-stack mirror set:
1. **Enumerate the mirror set** first — load the inline mirror index in the owning reference, or (if the index is incomplete) grep distinctive phrases for both the new corrected wording AND the old phrasing the fix replaces. A correction that removes a concept (e.g. replacing "fallback to baseline" with fail-closed) will not surface stale copies if you search only the new phrase; search the old phrase plus known inverse / synonym wording too.
2. **Reconcile each sibling against the invariant**, not against the corrected file. For each mirror, check whether its current wording states the invariant correctly; check old phrase, new phrase, known synonyms, and inverse/forbidden wording. A sibling can be silently inverted ("fail-open" where the invariant is "fail-closed") and still not match a naive grep of the new wording.
3. **Skip generated artifacts**: generated clients, generated docs, generator output trees (`kitex_gen/`, `hertz_gen/`, codegen output) update through their generator, not by hand. Constrain grep to authored references (use file-pattern scope) or filter generated paths.
4. **Update every authored copy in the same commit**, or document why a sibling deliberately diverges (the reason goes into the sibling reference itself, not just the commit message).
5. **Pre-commit verify on the actual staged tree, not the working tree**: ordinary `grep` / `rg` reads the working tree, so unstaged edits in sibling files can make verification appear clean while the staged commit still contains drift. Use index-aware search — `git grep --cached <phrase>` — or check out the staged tree to a temporary worktree and grep there. The search set must match the semantic discipline in step 2: new phrase + old phrase + known synonyms + known inverse / forbidden wording, not only the new phrase. A staged-tree search of one phrase still permits a stale sibling expressed via synonym or inverse wording to pass. The point is to inspect what the commit will actually contain, and to inspect it for every semantically-equivalent statement of the rule. Post-commit grep is a secondary confirmation, not the only proof — once committed without sync, the same-commit guarantee is already broken and needs `git commit --amend` or a follow-up sync commit. Amend only when the commit being amended is unpushed AND is the one just created — plus, **in a multi-batch same-branch program only**, one further condition: you discovered this drift independently. A sync fix requested by, or made to disposition, a review/challenge finding carries findings provenance whatever its shape, and inside such a program those take a NEW commit per `extraction-quickstart.md` 3g; outside such a program 3g does not govern, so a finding-derived sync there may amend on exactly the unpushed-and-just-created terms above. The provenance is recorded either way; only the NEW-commit consequence is program-scoped. Confirm with `git log -1` immediately before amending; once any later commit exists on the branch — the ordinary case once a multi-batch program is under way, since its findings fixes are NEW commits per `extraction-quickstart.md` 3g, though not at the very first commit — `--amend` would silently retarget that later commit, so the sync lands as a follow-up commit, never amend.
6. **For non-trivial fixes, the synced mirror set re-enters the dual-track gate**: a synchronization commit can introduce production-risk inconsistencies of its own (e.g. correct rule mirrored but with a subtly different timeout value, or correct policy mirrored to the wrong sibling). Run the next challenge round on the synced state, not on the single-file fix.
### Triggers — when this check is mandatory
- Any commit that fixes a **semantic** finding (review or challenge) in a rule with a documented sibling mirror. Wording-only edits to one copy of a mirrored rule (typo, grammar, formatting that preserves meaning) can be marked `mirror-impact: none` in the commit message and skip the cross-file step.
- Any commit that changes a **mirrored invariant** in a Cross-Stack Contract / Front-End Consumer Contract / Inference-As-RPC / Cross-Stack Conventions section. Producer-side and consumer-side rules are intentionally different in detail — synchronizing wording verbatim is wrong. The check is about the underlying invariant, not the surface wording: both sides must state the same direction of the same rule.
- Any commit that changes wording on a security-sensitive invariant (fail-closed vs fail-open, trust boundary, signed-credential semantics, lane isolation, cache key composition) where one ref is the architecture-layer statement and another is the test obligation.
- Any commit that touches a **likely-mirrored contract topic** even when no mirror index is declared. Organic mirrors exist throughout the skill tree (outbox / dual-write, idempotency keys, retry+CB pairing, observability label set, signed-URL policy, etc.) without an inline mirror declaration — the rule was reproduced across owning skills as the cross-stack contract emerged. Before changing such a rule, run a discovery grep: search distinctive phrases of the rule across the entire shared skill tree — every `SKILL.md` entrypoint plus every `references/` directory. Producer/consumer pairs and architecture/test pairs frequently land as **two-file** mirrors, so treat **two or more authored hits** as a likely organic mirror set even if undeclared. Reconcile per the discipline above. After reconciliation, add an inline mirror index in the owning reference so the next change does not need to rediscover.
### Anti-patterns
- "I'll fix the sibling in a follow-up" — the follow-up rarely happens; the drift becomes the new state.
- Trusting the commit message: a commit can claim "synced across files" while only changing one file. The pre-commit verify on the staged tree is the only proof; the commit-message claim is documentation, not enforcement.
- **Searching only the new wording**: a fix that removes or inverts a concept leaves stale copies that do not contain the new phrase. Grep both directions.
- **Reconciling to the newest commit instead of the invariant**: if the invariant is unsettled, do not propagate a tentative fix across siblings — resolve the invariant first (dual-track challenge, external spec, platform owner decision), then reconcile.
- Hand-editing generated artifacts to "make the grep clean": generated output updates through its generator.
- Not declaring the mirror set up front: when an extraction lands a Cross-Stack Contract for the first time, record the mirror set inline in the owning reference (which files now carry which side of which rule). Private-archive-only records have false negatives for readers who do not load the archive.
## Bounded Independent Review Packet
Use this when an independent review is required but a broad reviewer prompt hangs, returns no output, or starts expanding beyond the intended review scope. For non-wording extraction work, the gate-valid path is `scripts/extraction_review_gate.sh`, which makes each pass single-shot while delegating transport to the provider-neutral `code-review` controller. A strictly proven wording-only change uses the proof-bound generic single-review recipe in `code-review/references/staged-review-contract.md`; its controller-derived wording scope and independent `wording_only_boundary` result replace neither one another nor a failed semantic check. Both paths preserve the frozen packet, family exclusion, structured validation and client-specific recovery; raw provider CLI packets are debugging/advisory only and must not be recorded as passing review evidence.
Required flow:
1. Prove the applicable owner gate is available. For non-wording work, run `skills/skill-extraction-workflow/scripts/extraction_review_gate.sh --help`; for a strictly proven wording-only change, use the generic `code-review` help recipe. This local availability check does not invoke a model and is not review evidence. Do not use a raw provider "ping" as gate evidence. If the gate cannot run after documented remediation, record the lane as unavailable/inconclusive with command evidence.
2. Stop the stuck review process before retrying. Do not leave background reviewer sessions running and do not count a no-output process as a completed review.
3. Build a bounded packet from the exact changed files or excerpts being claimed. Prefer `git diff -- <files>` for local changes.
4. Feed a non-wording packet through `scripts/extraction_review_gate.sh`; feed a strictly proven wording-only packet through the proof-bound generic single-review recipe. The wording-only path uses its exact canonical full-context diff and proof file, stays untracked, and does not run `complete`; a custom/context-augmented packet or missing `wording_only_boundary` result re-arms challenge. In either case supply the actual implementer family, required `--review-plan-file`, stage, exact candidate, and explicit non-Claude egress approval when applicable. The plan binds the target, acceptance criteria, self-review and evidence before the independent run. The gate preserves tool posture, timeout, attribution and output parsing while following the user's local client order. A raw Claude packet may be used only to debug a wrapper failure and remains advisory:
```sh
git -C <repo> diff -- <files...> \
| claude -p --tools "" --no-session-persistence --output-format json "<review prompt>"
```
The review prompt must say:
```text
Review only stdin. Do not use tools. Do not request more context. Return findings ordered by severity. If there are no blocking issues, say no blocking findings.
```
5. Apply or explicitly reject actionable findings.
6. Rerun the owning skill validators and `git diff --check`.
7. Record the result as `findings applied`, `no blocking findings`, or `review unavailable after remediation` only when the wrapper or approved alternate produced valid structured evidence. Raw packet output is recorded separately as debugging/advisory and cannot close the gate.
8. Record each pass and the post-review delta per `references/dual-track-review-gate.md` (Recording the passes). A wording-only review records its single independent-review row.
Do not count as completed review:
- a hung command with no output,
- a broad prompt that may have scanned unrelated files,
- a prompt that allowed tools when the goal was diff-only review,
- a review packet that excludes files whose changes are being claimed.
This fallback reviews a landed diff or explicit review packet. It does not replace required Figma, code, document, or source reads for the extraction itself.
## Landing Checklist
- Keep the change scoped to the smallest owning skill.
- Back up or checkpoint existing skills before destructive rewrites, generated-artifact regeneration, or rollback.
- If a generator or writer owns the skill artifacts, update through that tool instead of hand-editing generated files.
- For shared installs, frontmatter `name` should match the installed command or directory name expected by the host; verify OpenCode, Codex, and Claude Code visibility separately when they are targets.
- For generated or installed skills, keep a small install/version trace when it helps debugging, but do not require manifest files for ordinary hand-authored skills.
- Commit with a message naming the capability added, not the source mined.
- Push when the user asks or the workflow requires shared availability.
- For shared local skills, verify install visibility for each target agent surface and keep local remotes free of embedded tokens or temporary credentials.
- If a later review finds overfit wording, fix it in a follow-up commit instead of silently rewriting history.
- If a validation step seeds or mutates a real tracked file (a test marker, a register seed, scratch state), undo only that exact addition and re-check `git status` before continuing — a blanket `git checkout`/`reset`/`rm` cleanup also discards your other uncommitted edits or deletes committed files (see `testing-strategy`).
- If this landing changes or reverses a rule a prior round landed, leave the old register row byte-for-byte untouched (append-only) — add the new row and put a `supersedes: <old row>` forward pointer on the NEW row's expected-behavior cell (not its Evidence cell; locate the old row by its rule text / round label, never by copying its old `*/SKILL.md` Evidence path), per `source-register.md` "Row lifecycle: supersede & revalidate" (never edit the old row or put `superseded` in `Status`; both break the impact-chain gate). If a landed rule's correctness depends on a versioned external surface that drifts (a specific tool/CLI flag, API/SDK version, vendor model name/tier, external standard part-year), tag it with a `revalidate-when: <event>` cue — an advisory re-check signal for the next round that touches this area, not an enforced loop.
- If review finds evidence overstatement, either add concrete source identifiers and observations that support the claim or downscope the wording before commit. Do not treat "no blockers after edits" as evidence that the original extraction was source-backed.
- If a correction reveals a workflow defect, land the workflow fix before claiming the corrected extraction is complete, or state why the defect is intentionally not being encoded.
- Do not say "tested again" or "pressure-tested" in the final response unless the validation record names the source artifacts reopened and the result classification.
- For stack-specific extraction, include the sibling-generalization mini-map summary in the final response or point to the durable source-map row that contains it. If the sibling map was reconstructed only after a correction, say that the original landing was incomplete and list the sibling decisions fixed now.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |