Skip to content

mikeparcewski/wicked-garden / skills/build-guide

wicked-garden-build-guide

Verified

Discover how a repo builds and tests, and write its build/test guide (`.wicked/build-guide.json`, contract v1): the data the wicked-core floor interprets instead of hardcoding runners. Reads CI config first, then the manifests, then each runner's own --version and --help. Prefers structured results (JUnit; JSON lines and TAP are reserved in v1) and writes a text parser, learned from this repo's real output, only when none exists. Writes a deliberately failing mutation probe, validates with `wicked-core floor probe` (only the probe writes `verified`), and proposes the file as a PR or governed change, never silently. Use when: "build guide", "write the build/test guide", "how does this repo test", "onboard this repo's tests for the floor", "re-onboard the build guide", "the build guide is stale", the onboarding step of the wicked-garden-core setup action, or a wicked-crew onboarding run dispatched because the floor found no guide, a stale one, or a failed probe.

FileKindSize
SKILL.mdskill md13.7 KB
assets/seeds/cargo-libtest.jsonconfig2.6 KB
assets/seeds/pytest-short-summary.jsonconfig2.5 KB
assets/seeds/vitest-junit.jsonconfig2.0 KB
refs/contract-notes.mdmarkdown12.1 KB
refs/discover.mdmarkdown4.9 KB
refs/propose.mdmarkdown1.9 KB
refs/results.mdmarkdown7.0 KB

refs/results.md

View on GitHub
# Results: structured first, a text parser only when there is none

## The id round-trip test (decides structured vs text)

A result format is only usable if the ids it reports are the ids the other pieces take. Ids
from `results` must appear in `list`, and they must work as `{id}` in `alone` and `deselect`.
The floor filters base failures through the base `list` and deselects them by id, so an id that
does not round-trip means no exclusion. Prefer the structured format that passes this test. If
the runner's structured ids do not round-trip, use the runner's own text summary, whose ids do.

**Worked example (pytest, JUnit vs text).** pytest's `--collect-only -q` list and its
`--deselect` take node ids (`tests/test_calc.py::TestGroup::test_x`). The best JUnit reading,
`-o junit_family=xunit1` with `results.id: ["file","name"]`, gives `tests/test_calc.py::test_x`.
That equals the node id for module-level tests, but not for class methods, and the `ids` probe
fails on it. pytest's short test summary (`-rfE`) prints `FAILED <node id>`,
which round-trips for every test. So the pytest seed is `text` on that summary.

## Structured kinds

- **junit**: put the reporter flag in `results.how.argvAppend` and the file in
  `results.how.path` (relative to `cwd`, under a scratch-only directory such as `.wicked-floor/`).
  The id is the `results.id` attributes (default `classname`,`name`) joined by `results.idJoin`
  (default `::`). Choose the attributes and the join that make the id equal to what `list`
  prints and `alone`/`deselect` take. A testcase with `<failure>` or `<error>` has failed, and
  one with `<skipped/>` neither passed nor failed (a name filter left it out).
- **json-lines** / **tap**: reserved in v1, because the interpreter reads only `text` and
  `junit`. Use JUnit, or text on the runner's summary.

## Writing a text parser

Write it only when the runner has no structured output, or its structured ids fail the
round-trip test.

1. **Capture real output** of this repo's runner with the mutation probe in a scratch copy
   (`rehearse.py`, or run it yourself). Write the rules against that text.
2. **Use the runner's own final summary** (and check it with more than one failure: one item
   that wraps onto a second line ends the block). Most runners end with a block that lists the failed
   ids (libtest's second `failures:` list, pytest's `short test summary info`). Express it as a
   `blocks` rule: `start`, `item` with `(?P<id>…)`, `end`, and `use`. Avoid per-line `failed`
   rules on lines interleaved with test output, because a test that prints a look-alike line
   would forge a failure. Use `failed` only when the runner prints nothing else.
3. **`use: last-before-end`** when the runner prints the same `start` line more than once
   before the summary ends: libtest prints `failures:` before the captured output and again
   before the list. Only the items after the last `start` count. **`use: all`** when every
   block is a real list.
4. **`section`** (optional): a line that opens a qualifier scope, with `(?P<q>…)`, for example
   a test binary or file header, when ids repeat across scopes. Set `expectQ` to match. When
   the runner prints that line on stderr, set `results.how.merge` (and `list.merge`).
5. **RE2 only.** No lookaround, backreferences, atomic or possessive groups (the interpreter is
   Rust). Every regex starts with `^` and ends with `$`, so that a full-line match and an
   anchored search agree. `check.py` enforces both rules.
6. **The ids the parser names must be the ids `list` prints.** Write `list.item` against the
   same captured run.
7. **Deselect composition** (it also applies to `list.run`, since the `deselect` probe
   deselects on the list). Tokens before the first `{id}` token are appended once (a leading
   `--` only when the argv has none). The tokens from the first `{id}` token through the last
   one repeat per id, and the rest are appended once. So put a flag and its value in **one**
   token (`--skip={id}`, `--deselect={id}`). With `"--skip","{id}"`, only the value repeats,
   giving `--skip a b`.

## Worked examples (also in `assets/seeds/`)

- **cargo / libtest** (`cargo-libtest.json`, reconciled with wicked-core's own probed guide):
  stable cargo has no JUnit/JSON (`--format junit` is nightly-only), so the test check is text.
  The CI `cargo build` step is a separate gate check (no `list`), like wicked-core's own
  `cargo fmt --check`, `cargo check` and `cargo clippy` gates. `version` is `cargo --version`. With
  `results.how.merge` and `list.merge`, the `section` reads libtest's `Running <binary>` lines
  (on stderr), so ids are qualified by test binary. The block runs from `^failures:$` to
  `^test result: FAILED`, with item `^    (?P<id>\S+)$` and `use: last-before-end`. `run` is
  the CI command exactly (`cargo test --locked --workspace --no-fail-fast`; drop `--locked` if CI
  does not use it). `alone` is `-- {id} --exact`. `deselect.append: ["--","--skip={id}","--exact"]`
  adds the separator once, `--skip=<id>` per id, and `--exact` once; libtest rejects a repeated
  `--exact`. The mutations are new integration test files
  (`tests/zz_floor_probe_mutation.rs`, `…_two.rs`). Each file is its own test binary, and so its
  own qualifier, so the first mutation file also holds a passing test
  (`floor_probe_mutation_beside`) for the `alone` probe. wicked-core's own mutations also arm a
  `ctor` hook its test harness needs, so copy a repo's own test preamble when it has one.
- **pytest** (`pytest-short-summary.json`): text on `-rfE`. The block runs from
  `^=+ short test summary info =+$` to the final count line, with item
  `^(?:FAILED|ERROR) (?P<id>.+?)(?: - .*)?$`. `list` is `--collect-only -q`, `deselect` is
  `--deselect={id}`, and `-p no:cacheprovider` keeps the scratch copy clean. When `CI` is set,
  pytest prints each failure's full, multi-line message in the summary. A continuation line ends
  the block, so only the first failure would be named. `results.how.argvAppend:
  ["--force-short-summary"]` keeps one line per failure (check that `pytest --help` lists the
  flag).
- **vitest** (`vitest-junit.json`): JUnit through `--reporter=junit --outputFile=…`, with
  `results.id: ["name"]` (the full name, `suite > test`), an `npm install` gate for the CI
  install step, and `version` `npx vitest --version`. That matches `vitest list`'s
  `file > name` line with item `^\S+ > (?P<id>.+)$`. The tests a name filter leaves out appear
  as `<skipped/>` testcases, which neither pass nor fail. vitest takes one name pattern only,
  and rejects a repeated one. So `alone` uses `--testNamePattern=^{id:re}$` and deselect uses
  one token for every id, `--testNamePattern=^(?!(?:{ids:re|})$)`. Both are regex-escaped, so a
  name with `(`, `.` or `+` stays literal.
- **A runner with no structured output** (the `tinyrun` fixture in the garden tests): a block
  from `^--- tinyrun summary ---$` to `^--- [0-9]+ failed, [0-9]+ passed ---$` with item
  `^FAIL (?P<id>\S+)$`. A per-line rule on `... FAIL` would be forged by test output that prints
  the same text.
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 323kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 323kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 323kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall