Skip to content

ericrisco/rsc-harness / skills/verify

verify

Verified

Use when work is about to be called done — the ASDD evidence gate, in two modes: per task inside `implement`'s loop (tests, build, security, traceability) and at closing over the whole branch against main (stack scripts/verify.sh, every done-check and acceptance criterion, rollback plan, dated verdict). A check that did not execute is reported not run, never passed. NOT judging the diff by eye (that is `review`, spec-less `code-review`), NOT diagnosis (that is `debug`).

FileKindSize
SKILL.mdskill md21.6 KB
evals/README.mdmarkdown4.2 KB
evals/cases.yamlconfig8.3 KB
references/generic-gate.mdmarkdown2.2 KB

evals/cases.yaml

View on GitHub
skill: verify

# Prompts that MUST load the `verify` skill. `verify` is the rsc-sdd
# post-implementation GATE: run the relevant stack scripts/verify.sh
# (lint/type/test/audit), walk every task done-check and every spec acceptance
# criterion, and produce an evidence-backed PASS/FAIL verdict before anyone
# claims "done". It does NOT write tests/features (implement), diagnose a
# failing check (debug), or read the diff adversarially (review).
should_trigger:
  - prompt: "I finished the orders feature — can you confirm it's actually done before I merge?"
    why: "Confirming 'done' before a merge is the gate's whole job: run the stack gate, walk the done-checks and acceptance criteria, and produce evidence."

  - prompt: "Run the lint, type-check and tests on this change and tell me if we're green."
    why: "Running the project's lint/type/test gate and reporting pass/fail is exactly what verify orchestrates via the stack scripts/verify.sh."

  - prompt: "Did all the acceptance criteria from the spec actually pass?"
    why: "Checking spec acceptance criteria against real evidence is a core WALK step of the verify gate, not implementation or review."

  - prompt: "Are we ready for review yet?"
    why: "The transition into the review phase is gated by verify; readiness means every check is green and every criterion is proven."

  - prompt: "Prove the fix works — I don't want to just take your word for it."
    why: "Demanding observable evidence rather than a claim is the skill's founding rule ('no evidence, no claim'); verify produces that proof."

  - prompt: "Walk every task's done-check and show me what's still open."
    why: "Enumerating task done-checks with evidence (met / not met) is the explicit WALK step verify performs and records."

  - prompt: "Before I say this is shipped, gate it: checks plus criteria, with the receipts."
    why: "A pre-ship gate that ties stack checks to acceptance criteria with recorded evidence is precisely verify, the phase that precedes review and ship."

  - prompt: "Coverage and the audit need to be green for this PR — can you check and record the result?"
    why: "Running coverage/dependency-audit through the stack gate and writing a verification record under 02-DOCS/wiki/sdd/ is verify's RUN+RECORD behavior."

# NEAR-MISS prompts that must NOT load `verify`. Each routes to a sibling that
# actually exists in this repo. The recurring trap: anything that WRITES or
# RUNS code, or owns the lint/test tooling itself, belongs to a stack skill or
# another phase — verify only orchestrates and judges.
should_not_trigger:
  - prompt: "Write the pytest cases and coverage config for this FastAPI service."
    route_to: "fastapi"
    why: "Authoring the actual tests and tooling is owned by the fastapi stack skill (which ships scripts/verify.sh); verify runs that gate, it does not write the suite."

  - prompt: "Our `go test -race` is failing intermittently — find out why and fix it."
    route_to: "go"
    why: "Diagnosing and fixing a flaky Go test is root-cause work owned by the go stack skill's testing guidance; verify reports the FAIL and hands off, it does not debug."

  - prompt: "Set up the verify.sh quality gate (eslint, tsc, build) for our Next.js app."
    route_to: "nextjs"
    why: "Building/owning the stack gate script itself is the nextjs skill's job; verify consumes that script, it does not create the stack's tooling."

  - prompt: "Audit this auth endpoint for injection and broken access control before we trust it."
    route_to: "security-audit"
    why: "A security audit of code is owned by security-audit; verify may invoke that audit as part of the gate but the hands-on review belongs to the security skill."

  - prompt: "Check that this migration applies cleanly and the CHECK constraint behaves as expected."
    route_to: "postgresdb"
    why: "Schema/migration and constraint verification is data-layer work owned by postgresdb; verify delegates to it rather than hand-rolling SQL checks."

# Concrete tasks verify should guide, with a rubric to grade WITH vs WITHOUT
# the skill loaded. A skill-guided answer runs the real gate, walks both lists,
# and refuses to call it done without evidence; an ungrounded answer tends to
# eyeball the code, trust stale results, or declare success prematurely.
capability:
  - scenario: >
      A change is finished and the user says "it's done, let's merge". Verify it, and produce the
      verification record.

      Everything you need is here — do NOT assume any of it exists on disk; this scenario is
      self-contained on purpose.

      APPROVED SPEC (acceptance criteria):
        AC1. Submitting an order returns 201 with the created order id.
        AC2. Submitting the SAME order twice creates exactly one order; the second call returns
             200 with the first order's id.
        AC3. An order with a negative quantity is rejected with 422 and nothing is stored.

      TASK DONE-CHECKS from the plan:
        T1. POST /orders creates an order.            done-check: a test asserts 201 + id.
        T2. Duplicate POST is a no-op.                done-check: a test asserts one row after two POSTs.
        T3. Negative quantity is rejected.            done-check: a test asserts 422 + no write.

      WHAT THE IMPLEMENTER REPORTS:
        - Files changed: app/orders.py, tests/test_orders.py
        - "I ran the suite: 2 passed. Tests written: test_create_order_returns_201,
           test_negative_quantity_rejected."
        - "mypy wasn't installed so the type step printed SKIP, and coverage ran without a
           threshold configured. Everything else was green."
        - "The retry loop can spin forever" was raised in review; the implementer replied
          "not a real problem".
    must_include:
      - "Reads technical_level from 02-DOCS/wiki/harness/user-profile.md and picks the register (verdict unchanged by it); reports short, with one line of why per failing check."
      - "LOCATES the three acceptance criteria and the three task done-checks from the material given, and maps each to what the implementer actually reports."
      - "Refuses to accept the implementer's summary as the gate: says the checks must be re-run and the output read, rather than trusting a reported '2 passed'."
      - "Treats the mypy SKIP as UNVERIFIED rather than a pass, and does not let 'everything else was green' stand in for the skipped layer."
      - "WALKS all three done-checks and all three acceptance criteria, and catches that AC2/T2 (duplicate POST is a no-op) has NO test at all — the two named tests cover AC1 and AC3 only — marking it UNVERIFIED rather than passing it."
      - "Does NOT fix a failing test inline; if a check fails it records it and hands off to debug, and refuses to fix-then-claim."
      - "RECORDS a dated verification file under 02-DOCS/wiki/sdd/verifications/ and indexes it in 02-DOCS/wiki/index.md (never the root CLAUDE.md)."
      - "Returns a VERDICT that is PASS only if EVERY item has passing evidence; a single unverified acceptance criterion makes the whole verdict FAIL — and refuses to call it 'done' otherwise."
      - "On PASS, points to the review phase; on FAIL, routes failing checks to debug and missing coverage back to implement."
      - "Treats a coverage GAP (the layer ran with no threshold configured, so it could not have failed) as an unverified criterion that fails the verdict — not as a pass — and records it as SUSTITUIDA: a report in place of a gate, unable to detect coverage regressing."
      - "Before trusting any home-grown gate that reports green, asks for it to be watched failing on a known-bad input, and states the limit: one negative control proves one bad case reaches the failure path, not that the gate covers the rule it claims to enforce."
      - "Splits layers that did not run into NO-APLICA / HERRAMIENTA-AUSENTE / SUSTITUIDA rather than one undifferentiated SKIP list, and never writes SUSTITUIDA as a pass."
      - "Binds the record to a source state (short commit SHA plus clean/dirty) and takes every number from one fresh run made after the last edit, refusing to reuse mid-task figures."
      - "Lists every check that did not execute (the mypy SKIP) as 'not run', never as passed, and writes a rollback plan into the closing record."
      - "Gives any dismissed finding its own evidence line (command, file:line, or the test that disproves it) rather than asserting it was not a real problem."
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall