Skip to content

lidge-jun/codexclaw / plugins/codexclaw/skills/qa

cxc-qa

Verified

Use for manual web, TUI, CLI, API and desktop QA. Triggers: smoke test, visual QA, screenshot check, CJK clipping, 수동 QA, 실제로 되는지 확인, 동작 확인, 직접 돌려봐.

references/evidence-contract.md

View on GitHub
## 3. Evidence contract

Artifacts live under `<evidence root>/<sessionId>/qa/<scenario-id>/`, where the evidence
root is `.codexclaw/evidence/` in repo mode and `<durable>/evidence/` from `cxc state path` otherwise:

- `invocation.txt` — the exact command(s)/steps, copy-pasteable.
- the artifact(s) — capture, screenshot, response, transcript.
- `verdict.json` — `{ "scenario": "<id>", "criterion": "<what this proves>",
  "surface": "http|cli|tui|web|gui", "verdict": "PASS|FAIL|NA",
  "artifactRefs": ["<relative paths>"], "note": "<one line>",
  "capturedAt": "<RFC3339>", "sourceSnapshotAt": { "kind": "resolved|unavailable",
  "commitSha": "<sha>", "dirty": <bool>, "capturedAt": "<RFC3339>",
  "treeHash": "<sha256, dirty only>" } }`.
  On `web` and `gui` only, also `"captureChecks": { "signature": <bool>,
  "nonEmpty": <bool>, "dimensionsMatch": <bool>, "composited": <bool> }` —
  all four keys (QA-CAPTURE-INTEGRITY-01 in `visual-qa.md`).
  A desktop row that depends on a built artifact also sets
  `"desktopArtifact": true` and `"criterionIds": ["c-3"]` (its goalplan
  criteria) and lists exactly one `artifact-identity.json` in `artifactRefs`.
  The identity schema is DESKTOP-ARTIFACT-01 in cxc-dev-devops
  [native-desktop-acceptance.md](../../dev-devops/references/native-desktop-acceptance.md).

Rules:

- Every PASS names at least one non-empty artifact in `artifactRefs`.
- `capturedAt` and `sourceSnapshotAt` are required on every surface: knowing
  when evidence was captured, and against which tree, does not depend on what
  kind of surface produced it (QA-EVIDENCE-FRESHNESS-01).
- `inferred` and `partial` verdicts DO NOT EXIST — a scenario either ran
  against the real surface or it did not.
- A scenario that cannot run is a FAIL carrying the blocker and the missing
  prerequisite, not a skip.
- `NA` is legal only when the class structurally cannot apply to the surface
  (e.g. viewport class on a headless API), always with a recorded reason.
- This directory is shared with the SubagentStop receipt gate's root
  (`<durable>/evidence/`, using the durable directory shown by `cxc state path`); main-session QA artifacts do not interact with
  worker receipts (the gate validates only the worker's own
  `EVIDENCE_RECORDED:` marker path).
- A worker that CANNOT write there was dispatched wrong, not gated wrong: read-only
  lanes belong on `agent_type:"explorer"`, which the gate never touches. The retry
  budget is terminal, so a mis-routed worker is released rather than trapped — but its
  verdict stays unresolved and blocks goal completion until a receipt settles it.

After every scenario is done, emit the aggregate receipt. Replace `<evidence-root>` with
the resolved durable directory plus `/evidence`:

```
node plugins/codexclaw/skills/qa/scripts/validate-evidence.mjs \
  "<evidence-root>/<sessionId>/qa/" --emit-receipt
```

It validates every `verdict.json`, confirms they all describe the same tree,
and writes `<evidence-root>/<sessionId>/qa-receipt.json`. Validated verdict
and identity files are hashed into typed `artifactManifest` entries
(`path`, `sha256`, `kind`, optional `criterionIds`), rechecked whenever the
receipt is read. A schemaVersion 2+ plan with a recorded final gate needs an
identity entry for each non-native desktop criterion; default v1 plans rely on
this ingress validation only. Any failure
leaves no receipt behind, including deleting one an earlier run produced —
a receipt that outlives the QA it attests to is worse than none.

Current limitation: the final gate does not pick this file up on its own. Until
the `final-gate` lifecycle CLI exists, point `finalGate.qaReceiptPath` at that
path by editing the goalplan directly. Remove this paragraph when that CLI lands.
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall