DevNullLtd/Yuzu / .codex/skills/test
test
UnverifiedRun or plan Yuzu's `/test` pre-commit and pre-push validation pipeline with quick, default, full, instructions, and quarantine modes. Use when the user says `/test`, `/test --quick`, `/test --full`, asks for the full test gate, or needs the Yuzu test-runs DB workflow.
Test
Use this skill for the high-level Yuzu validation pipeline. Route low-level build choices through $yuzu-build, Windows through $yuzu-windows-msvc, and Erlang caveats through the gateway guidance below.
Modes
- default: build, upgrade test, live UAT, standard gates. About 30-45 minutes.
--quick: build plus offline C++ unit, EUnit, and Dialyzer. No live stack, no synthetic UAT.--full: default plus OTA stubs/flows, sanitizer dispatch, coverage, and perf measurement. About 60-120 minutes.--instructions: live-stack instruction definition suite only.--instructions-quarantine: local quarantine ceremony only. Do not run on remote or SSH-only hosts.--force-cleanup: clean this run's dangling containers before start.--keep-stack: leave debug stacks running when the script supports it.
Run State
Every local /test pipeline run records gate results and timings in
~/.local/share/yuzu/test-runs.db, overrideable with YUZU_TEST_DB. Query with:
bash scripts/test/test-db-query.sh --latest
bash scripts/test/test-db-query.sh --last 5
bash scripts/test/test-db-query.sh --diff RUN_A RUN_B
bash scripts/test/test-db-query.sh --trend timing=phase7.perf
bash scripts/test/test-db-query.sh --flaky
The self-hosted CI pools also keep a persistent database per runner agent; they do not share one host-wide SQLite file:
- Big Tam:
/srv/ci/work-N/_tool/yuzu-test-runs/yuzu-bigtam-linux-N/test-runs.db - Wee Tam:
D:\ci\test-runs\yuzu-weetam-windows-N\test-runs.db
ci.yml's Linux and Windows jobs initialize and write these databases
automatically. Schema v3 preserves GitHub rerun attempts separately and stores
job outcome/duration, platform, runner, commit/branch/event, ccache hit ratio,
Meson suite result/duration/timeout, and recovered known-flake events. Query one
on its host by setting YUZU_TEST_DB, for example:
YUZU_TEST_DB=/path/to/runner/test-runs.db \
bash scripts/test/test-db-query.sh ci-stats --since 30d
YUZU_TEST_DB=/path/to/runner/test-runs.db \
bash scripts/test/test-db-query.sh ci-suite-stats --since 30d
YUZU_TEST_DB=/path/to/runner/test-runs.db \
bash scripts/test/test-db-query.sh ci-flakes --since 30d
Provision/repair paths are deploy/linux/Provision-BigTam-Runner-Telemetry.sh
and deploy/windows/Provision-Windows-Runner.ps1. GitHub-hosted macOS runners
are ephemeral and therefore cannot satisfy the runner-local persistence
contract; their Meson logs remain Actions artifacts. ci-ingest is a coarse
workflow-level backfill for older GitHub history, not a replacement for direct
runner recording.
tests/known-flaky.json remains the reviewed allowlist. A listed case only
passes CI if an isolated retry recovers; each recovery is also persisted in
ci_flake_events so recurrence can be measured instead of inferred from logs.
Initialize each manual orchestration with a RUN_ID, TEST_DIR=/tmp/yuzu-test-$RUN_ID, LOG_DIR=$HOME/.local/share/yuzu/test-runs/$RUN_ID, COMMIT, BRANCH, and BUILDDIR=$(build_dir) from scripts/test/_portable.sh. Record gates with scripts/test/test-db-write.sh.
Phase Order
Wall-clock order is fixed:
- Phase 0 preflight:
bash scripts/test/preflight.shor--force-cleanup. - Phase 1 build HEAD: Meson compile, Erlang
rebar3 as prod release, and docker images except in quick mode. - Phase 2 upgrade test:
bash scripts/test/test-upgrade-stack.sh ...; skipped in quick mode. - Phase 3 OTA: full mode only; may record SKIP if still stubbed.
- Phase 7a perf: full mode only, before UAT stack; measure-and-report only.
- Phase 4 fresh stack:
bash scripts/start-UAT.sh; skipped in quick mode. - Phase 5 gates: unit, EUnit, Dialyzer, CT, integration/e2e/synthetic UAT/puppeteer/instructions as applicable. Security E2E runs after the parallel fan-out, alone (its rate-limit probe empties the shared login bucket). Puppeteer records SKIP when
tests/puppeteer/node_modulesis absent. - Phase 6 sanitizers: full mode only, dispatched runner.
- Phase 7b coverage: full mode only, enforces
tests/coverage-baseline.json. - Phase 8 teardown and summary.
Phase 1 is mandatory-blocking. Other phases should normally continue so the operator gets one prioritized report.
UAT Lifetime Rule
Phase 8 must not stop the native UAT stack from scripts/start-UAT.sh. A successful default/full run intentionally leaves a working stack at tested HEAD for human inspection. Only stop it if the user explicitly asks.
Erlang Gateway Rules
- Source
scripts/ensure-erlang.shbefore directrebar3work, then verifycommand -v erl. - For EUnit, use
rebar3 eunit --dir apps/yuzu_gw/test; the--dirflag is mandatory. - Do not treat
rebar3 eunit --module Xas isolated; it runs after the full--dirphase in the same VM. - In parallel test fanout, use separate
REBAR_BASE_DIRvalues for EUnit and CT to avoid_buildraces. - Always run Dialyzer after Erlang changes.
Perf Status
Perf is measure-only as of 2026-05-03. scripts/test/perf-gate.sh records metrics and exits PASS; humans inspect trend and distribution shape using the test-runs DB and docs/perf-baseline-calibration-2026-05-03.md.
Mode Shortcuts
For --quick, resolve the build directory and run only compile plus offline suites. Do not start UAT and do not invoke synthetic UAT.
For --instructions, use scripts/test/instructions-tests.sh after bringing up only the required live stack.
For --instructions-quarantine, use scripts/test/instructions-quarantine.sh only on a local machine where temporary self-disconnect is acceptable.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |