ccoalm/ccl-skills / skills/skill-extraction-workflow
skill-extraction-workflow
Flagged复盘 / 沉淀 / 总结经验 / 补进技能 / 技能缺陷 / 流程优化 / would other teammates hit this / will we forget next time / teamwide recurrence / extract skill / post-mortem / 深度 review·审计技能仓库 / 对标外部技能包找 gap(benchmark superpowers·gstack 等) / 核查全局安装点(~/.config/opencode 等)旧快照是否遮蔽本仓技能 / 本仓(ccl-skills 等共享技能仓)OpenCode 项目配置·命令治理 → turn reusable lessons into durable CCL skills.
references/coverage-exhaustion-traps.md
View on GitHub# Coverage-exhaustion traps — worked failure shapes
Companion to the **`digest-masks-corpus trap`** Core Rule in `SKILL.md` (under *Evidence, RCA, charter & attribution*). That Core Rule carries the **gates + triggers** (when an "exhausted / complete / no-gap / fully-extracted" claim is unsupported, and what closes it). This file carries two things: the **relocated gate detail for variants (a0) and (4)** — the executable obligation text moved here from the Core Rule so the entrypoint stays at gate altitude (the Core Rule keeps the variant names and a pointer) — and the **recurrence-memory** for the core form and variants (a)/(b)/(c): the concrete historical failure shapes that justify each gate, plus the fuller worked examples. Read it when you are about to make such a claim and want the concrete shape of how each variant goes wrong, or when you need the (a0)/(4) obligation text.
The trap has one core form and variants (a)/(b)/(c) as failure shapes; the Core Rule's walked enumeration adds (a0) and standing gate (4) to the same fix family, and this file carries their relocated gate detail. The core form and variants (a)/(b) share one fix — **enumerate the real underlying artifact and mark each unit `read`/`skipped`, rather than trusting the digest**. Variant (c) is the case where *no artifact masked anything*: the **source-class universe itself was never enumerated**, so its fix is to enumerate the class taxonomy instead.
## Core — digest exhaustion ≠ repo exhaustion
The digest (a high-quality maintainer `AGENTS.md`/`CLAUDE.md`, README, CONTRIBUTING, or architecture/design doc) and the underlying code are distinct source classes; a strong digest masks how much code went unread.
**Failure shape:** deep-read a 50KB `AGENTS.md`, land its lessons, write "extraction exhausted" while ~hundreds of KLOC are unread, and a user has to ask "你没看代码么 / did you read the code".
Note on depth: the code-corpus sweep is usually inventory + owner-mapping + novelty-spot depth, not line-by-line — but it must be a **closed register row**, and is commonly low-yield because the digest already distilled the reusable layer. Record that outcome honestly rather than skipping the row.
## Variant (a) — self-retrospective form (the digest is your OWN output)
In a task / session / multi-batch retrospective the same trap fires with the agent's own artifacts as the digest: prior retro summaries, MR / commit *descriptions*, and project-memory notes *about* the work are a lossy digest of the actual change set, not the change set. Reading your own summaries back and finding nothing new is NOT evidence the diffs hold nothing — and a fact recorded only at the project-memory layer is not the same as a lesson promoted to the owning shared skill.
**Failure shape:** declare a multi-repo program "fully summarized" from MR descriptions and self-notes while the real diffs still carry reusable lessons (a locale-stable manifest-sort fix, a per-surface-redaction refinement) that were noted at the project layer but never promoted, and a user has to ask "根据实际变更核查 / 不要只看 comment".
## Variant (b) — benchmark / no-gap form (the conclusion is "we already cover this")
When the source is an external artifact you are benchmarking to decide whether WE must extract, a "no gap / already covered / it converges with what we already have" conclusion is ALSO a coverage claim under this trap. The digest here is the source's **overview / README / feature-docs / commit-messages — prose written to *advertise* a feature** — which systematically omits the very safety mechanics worth borrowing. Concluding "no new lesson" requires reading the source's **load-bearing decision surface**: the actual decision rules, the claim-vs-ground-truth checks, the data-loss guards. Where that surface is code separate from the docs, read the code; where the load-bearing artifact is *itself* executable prose (a skill's own `SKILL.md`, a standards/spec doc with no code), read THAT rule surface in full — not the overview of it — and scope the no-gap claim to the artifact actually read.
**Benchmark source classes include agent-native product repositories, not only skill packs.** When the benchmark target is "how does the outside world do this", enumerate at least: published skill/prompt packs (`superpowers`, `gstack`, vendor packs); *and* product repositories developed largely by agents that ship their development discipline in the tree (root/subtree `AGENTS.md`, agent decision-record and note conventions, repo-local skills, testing policy, pre-push/evidence-selection rules, postmortem and defensive-pattern docs). The second class is where operating discipline lives; a benchmark that only reads skill packs mistakes the absence of a competing skill for the absence of practice. Read such a repository's discipline layer as its own register row (digest `AGENTS.md`/README first, then the notes/skills/policy corpus — the digest-masks-corpus rule applies twice over), classify its portfolio stability before borrowing, and land what you borrow as "one industry shape" unless independently confirmed.
**Failure shape:** "this external self-maintenance design is already covered, nothing to extract" claimed twice from its feature-docs + commit messages, until a user said "真实看对方实现逻辑 / read the actual implementation" and the code revealed two real gaps (a claim-vs-ground-truth reconciliation and a package-integrity guard) the docs never surfaced.
## Variant (c) — channel-taxonomy form (nothing masked it; the class set was never listed)
The three shapes above all have a **digest standing in front of a corpus you could already reach**. This variant has no digest and no corpus-in-hand: the claim is that an *acquisition channel set* is exhausted, and it fails because the **taxonomy of channel classes was never enumerated** — so the agent kept searching harder *inside* the two or three classes it had thought of. The tell that separates it from the core form: each round surfaces a **class the agent had never considered**, not another artifact within a known class.
Its fix is therefore not "read the real artifact" but **walk a seed list of channel classes to catch the ones never considered, and then scope the claim to what was actually walked** — the seed list is not a universe, so it licenses no exhaustion claim; what makes the output falsifiable is recording each class's search boundary and which classes were not walked, because otherwise "I could not find more" and "there is no more" are indistinguishable. Closure and downgrade belong to the owning skill's existing coverage gate, not to a status vocabulary invented here.
**Failure shape:** an agent researching one organization from public sources declares "the public channels are mined out" five separate times; each time the user pushes back, a **previously unconsidered channel class** lands major evidence (an archive route around a bot-block, a regulator's bulk dataset, a mandatory-disclosure regime, another jurisdiction's statutory filings, a filing the organization voluntarily publishes on its own site). No artifact was hiding these — they were never on a list. The domain-side channel taxonomy lives with the research owner (`skills/multi-perspective-research/references/public-disclosure-channels.md`, repo-root relative); what belongs here is the recurrence memory that an exhaustion claim over a *channel set* needs a walked class list, not more effort inside the known classes.
## Variant (d) — the OTHER agent line's store (the corpus you cannot see from where you stand)
More than one agent runs against the same repository, and each keeps its lessons in its own host's store. Those stores do not see each other. An extraction that enumerates "the lesson corpus" from the host it happens to be running on has enumerated **one line's half of it** and will report exhaustion over a corpus it never touched — the core trap, with host boundary as the thing doing the masking.
Measured in round 059: one line held hundreds of `symptom -> cause -> fix` records while another held a few dozen distilled entries, in different trees under different roots with no reference between them. The figures are deliberately not pinned here — these stores are LIVE and grow while you work, so a count written into a durable document is a timestamped measurement that starts rotting the moment it lands; a re-count during this round's own self-check already disagreed with every figure it had recorded hours earlier. State DEPTH, which is stable, not SIZE, which is not. Neither line's store contained a pointer to the other. The consequence is not abstract: one class recurred **eight times across independent sessions** on the line that lacked the abstraction, while the line that had already generalized it — same repository, same weeks — carried the remedy the whole time.
The gate: before any complete / exhausted / no-gap claim over a *lesson or failure* corpus, the landing that makes the claim carries, **for each agent line, a verdict** — `covered` / `digest-only` / `unavailable` / `excluded` — and under it one row per store backing that verdict. A per-store list without a per-line verdict is the failure this gate exists to stop: a reader cannot tell from four mixed rows whether the line was actually covered. A line whose store you cannot read is `unavailable` with the remediation attempted, never an omission; a line you deliberately leave out is `excluded` with the reason. Locate the stores by enumerating the hosts that actually work the repository, not by assuming the layout your own host uses — the other line's may be one large per-session file where yours is a directory of entries, and a path-shaped guess finds nothing and reads as absence (the same "keyed by something other than what you are asking about" failure the blocked-source ladder names).
**Where these rows live — the split that makes the gate checkable.** Two different things get confused here. The *provenance* — the corpus itself, and the project, repository, branch, ticket and contributor identifiers inside it — is what the extraction lifecycle keeps in per-host scratch, and it stays there. An agent product name and its own config path (`~/.codex/memories/`) are neither: they identify a tool this repository already names throughout, carry nothing about any project, and a leakage audit over them comes back clean. The *coverage rows* are store location, status, depth and exclusion reason; they carry no corpus content, so nothing requires them to be private, and holding them in scratch makes the gate unverifiable by the only people who will ever read the claim. So: coverage rows go **where the claim is made**, alongside it and in the same artifact; provenance stays in scratch and the coverage rows point at it by role, never by real path. If a line's store cannot be described without naming something private, that is a sanitization problem to solve in the row, not a reason to move the row. The round that introduced this variant records its own, so the rule is demonstrated rather than asserted:
**Codex line — verdict: `covered`.** One store was read to the bottom, and what was left out is left out for a stated reason, not by omission.
| store | status | depth actually read |
| --- | --- | --- |
| `~/.codex/memories/MEMORY.md` | `deep-read` | every session block expanded; every failure bullet read individually, not sampled |
| `~/.codex/skills/.extraction-work/` | `read` | all artifacts present at the time, charter/closeout fields |
| `~/.codex/sessions/` | `excluded` | not read — the distilled memory file above is already their `symptom -> cause -> fix` reduction |
| `~/.codex/memories/raw_memories.md` | `excluded` | not read — same source as the distilled file, which is structured |
**Claude Code line — verdict: `digest-only`.** Deliberately weaker, and the weakness is the point: nothing in this round may rest on this line alone.
| store | status | depth actually read |
| --- | --- | --- |
| `~/.claude/projects/<proj>/memory/` | `digest-only` | index lines only; entry bodies not expanded |
| `~/.claude/skills/.extraction-work/` | `read` | all artifacts present at the time, charter/closeout fields |
**Enumerate the line set before you fill the rows, and record how.** A table of two lines cannot reveal a third that was never listed — the gate false-greens on omission exactly the way the parent trap does on an unenumerated corpus. So the line set is *derived from something observable*, never recalled: list the agent home directories present on the machine (`ls -d ~/.*/ ` and pick the ones carrying agent state), then add any host named in this repository's own tooling and routing that has no directory yet. Record the enumeration you ran next to the verdicts, so a reader checks the SET first and the rows second. This is the channel-taxonomy failure of variant (c) applied to agent lines: nothing masks the missing line, it was simply never in the list.
The worked example below was enumerated that way, and the enumeration immediately paid for itself: run against the machine that produced this round it returned **five** agent directories, not the two the round had been working with. Three had been invisible to an author who was recalling rather than listing. Two of them (`~/.gemini`, `~/.copilot`) hold configuration and installed skills only — no session or lesson state — and drop out on evidence. The third (`~/.cursor`) is a real line carrying substantial chat state, and it earns an `excluded` row with a reason rather than silence. That is the whole point of the gate: without the enumeration the round would have claimed cross-line coverage over a set it had never established.
**Name the lines and their stores.** An anonymised table (`line A`, `line B`, "a memory file") cannot be audited: a reader cannot tell which agents were enumerated, cannot spot a third line that was never listed, and cannot check a store themselves — the gate false-greens on its own example. The private part is the corpus and the project identifiers in it, not the name of the agent that produced it. Name them, and let the sanitization rules do their job on the corpus rows instead of blanking the index.
**Cursor line — verdict: `excluded`, with a reason.**
| store | status | depth actually read |
| --- | --- | --- |
| `~/.cursor/chats/` | `excluded` | not read — raw conversation state with no distilled lesson artifact anywhere under the tree, the same class as the raw transcripts excluded on the Codex line |
| `~/.cursor/agents/` | `excluded` | empty |
**`~/.gemini`, `~/.copilot` — not lines for this purpose.** Configuration and installed skills only; no session or lesson state. Recorded here because "it turned out to hold nothing" is a finding, and leaving them off the list is how the next round re-discovers them.
Two things that table makes visible and prose did not. The lines are **asymmetric by store, not by sampling**: one keeps its lessons in the memory file and the other in the extraction-work directory, and the directory named the same on both holds nine times more on one line than the other. And a same-named path means different things per line, so a lookup shaped by your own layout returns nothing and reads as absence.
Do NOT resolve this by building a sync or mirror between the stores. Two independently-owned stores with a one-time cross-distillation is the shape that has held here; a mirror adds a mechanism to maintain and drifts silently the first time it is not run. The obligation is coverage at extraction time, not continuous replication.
## Variant (a0) — produced-artifact + next-run-delta (relocated gate detail)
**Firing point: for a task/session retrospective, this variant fires at CHARTER time — the produced-artifact row (i) enters the charter's evidence plan as its FIRST source class (see the Evidence plan field in `source-to-skill-extraction.md`), while the next-run-delta row (ii) is a separate required charter/closeout record, not a source class — not only when an exhausted/complete claim is about to be made** (the original anchor, kept as backstop). Observed failure shape of the late anchor: a 7-session program retro that never claimed "exhausted" walked past this gate entirely, took only correction turns as evidence, and landed zero method/craft lessons while r-series reports and a benchmark corpus sat unread in the project.
When the source is a task/session that PRODUCED runnable artifacts (scripts, commands, queries, configs, fixtures), two rows are required and neither substitutes for the other: **(i)** the produced artifacts are enumerated as their own source class in the **source register** (existing status vocabulary — `read`/`deep-read`/`excluded`/`routed`, with "project-specific only" recorded as the *reason* on an `excluded` row, never as a new status), and each reusable technique found in them gets a row in the **provenance-to-target diff** under its existing vocabulary (`landed`/`routed`/`downgraded`/`pending`) — **attaching the artifacts to a deliverable, or archiving them in the project, is NOT `landed`**, because a bundle hanging off one document is unreachable from the next project; and **(ii)** a **next-run delta** row in the source register whose fields are: what this run's cost was actually spent on, what the next run of this task shape must do IN WHAT ORDER, and which prerequisites must hold regardless of order — a sequencing lesson is a counterfactual about ordering, not an artifact, so the per-artifact sweep structurally cannot surface it. The framing trap this closes: extraction defaults to asking "what rules did we learn" (declarative) and never asks "what would we do differently, in what order" (procedural), so a session whose most reusable assets were working code and a better sequence lands only prose rules and passes every gate.
## Gate (4) — the per-item verdict-disposition obligation (relocated gate detail)
The verdict owes a **per-item disposition row EVEN WHEN THE ROUND LANDS NOTHING** (no commit fires to catch an advertising-level verdict): each row records the load-bearing surface actually read (section/line/rule — NOT headers/`When to invoke` blurb/feature-list), the owning organization artifact (`file:rule`) OR `route-to-tool` / `out-of-scope: <reason>`, and for `covered` the mechanical firing path. A `quick`/`light`/`sweep`/`triage` framing does **NOT** waive the load-bearing read. Record the row on a **concrete surface** — the triage turn's final response OR persistent scratch / `source-register`, never chat-ephemeral. (Honesty: absent a host/review hook checking that surface, this stays recognition-dependent — salience, not mechanical enforcement.) A repeated user "深度分析了么 / did you deep-read" across rounds is the recurrence signal that an advertising-level verdict shipped; treat it as a validation-gate defect.
## Gates (1) and (2) — the unit lists (relocated gate detail)
The Core Rule states the two gates; this is the detail the entrypoint no longer carries.
- **Gate (1), the terminal-status row.** The code-corpus row's terminal status is one of `deep-read`, `inventory+owner-mapping`, or a downscope that cites an actual user instruction — a self-declared downscope is not terminal. The depth note under *Core* above governs what that sweep usually looks like and why its outcome is recorded rather than skipped.
- **Gate (2), the enumeration unit.** "Top-level structure" means the artifact's own next unit, chosen so every part is either `read` or `skipped`: for a document its `##`/`###` sections, or TOC / pages / anchors / line-chunks when sections are absent; for a repository its top-level directories, or package / module / test / script / config units when the top level is flat. The trap recurs even within one artifact — a doc read to its third section with the remaining sections un-enumerated supports no exhausted claim over the doc.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |