Skip to content

ericrisco/rsc-harness / skills/implement

implement

Verified

Use when an approved plan and task list exist and it is time to turn them into working, reviewed code — ASDD stage 4, the per-task loop after `analyze`: TDD build and self-check, `rsc risk` on the diff, review scaled by tier (0/1/3 reviewers), mandatory controls (tests, build, security, traceability), at most 3 fix rounds, then fix-and-record or park in the decision queue. Progress ledger survives compaction; disjoint tasks fan out via `parallel`. NOT spec writing (that is `specify`), NOT planning (that is `plan`), NOT the closing whole-branch gate (stage 5: `verify` + `review`).

FileKindSize
SKILL.mdskill md24.4 KB
evals/README.mdmarkdown3.1 KB
evals/cases.yamlconfig5.1 KB
references/per-task-review.mdmarkdown4.3 KB
scripts/review-packageexecutablescript1.6 KB
scripts/sdd-workspaceexecutablescript1.3 KB
scripts/task-briefexecutablescript3.4 KB

evals/README.md

View on GitHub
# Eval harness — `implement` skill

Run by an **agent harness** (an LLM agent with the Skill tool), not a pure shell script.
`cases.yaml` is the fixture; this README is the honest run procedure. Three things to measure:
**triggering** (should fire), **anti-triggering** (must not fire), and **capability** (does the
skill make the build better).

`implement` is a process skill — it ships no `scripts/verify.sh`. It is judged on its safety rails
(test-first ordering, per-task checkpoints, decision logging, constitution adherence), not on a
deterministic command.

## 1. Triggering accuracy

For each prompt in `should_trigger` and `should_not_trigger`:

1. Start a fresh agent session with the `implement` skill available in the catalog, plus its named
   routing siblings (`verify`, `tasks`, `debug`, `worktrees`, and the stack skills such as
   `fastapi`) present but not the subject — so routing is realistic.
2. Feed the prompt verbatim. Do **not** hint that a skill exists.
3. Observe whether the agent invokes the `implement` skill.
4. Run **3–5 trials** per prompt (LLM routing is stochastic); record the fire rate.

Pass conditions:
- Every `should_trigger` prompt fires `implement` in **≥ 90%** of trials.
- Every `should_not_trigger` prompt does **not** fire it in ≥ 90% of trials, and the agent prefers
  the named `route_to` sibling. The boundaries to watch: `tasks` (makes the list, doesn't build it),
  `verify` (the final whole-suite gate, not per-task tests), `debug` (mysterious failures), and the
  stack skills (concrete test tooling). A near-miss leaking into `implement` is a harder failure
  than a missed trigger — fix the description boundary before shipping.

## 2. Capability uplift (with vs without)

For the `capability` scenario:

1. **Without skill:** run the scenario in a clean session, no `implement` skill loaded. Capture the
   answer.
2. **With skill:** run the same prompt with `implement` (and the named stack sibling) loaded.
3. Grade both against the scenario's `must_include` checklist — one point per bullet that is
   *concretely* present (e.g. an actual failing test shown first, not just the phrase "test-first").
4. Use 3 trials each side; average the coverage.

Pass conditions:
- **With** the skill: `must_include` coverage **≥ 80%**.
- **Measurable uplift:** with-skill coverage beats without-skill. The expected gaps in the
  without-skill run are the ones this skill exists to close: writing the test *after* the code (or
  skipping the red step), barrelling through multiple tasks with no checkpoint, hand-rolling a test
  runner instead of delegating to the stack reference, omitting the decisions.md trail, and claiming
  "done" without tying it to the done-check + acceptance criterion.

## What a failing eval tells you

- Trigger misses → the description's "Use when…" triggers are too narrow; widen the phrasings.
- Near-miss leaks → the boundary against `tasks`/`verify`/`debug` is blurry in the description; sharpen
  the NOT clauses.
- Capability gaps → the SKILL.md body under-specifies the loop (red→green→refactor), the checkpoint,
  the delegation, or the decision log; tighten the relevant section.
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall