Skip to content

wado-lang/wado / .claude/skills/coverage-investigation

coverage-investigation

Verified

Investigate and improve code coverage for the wado-compiler crate. Use for any question about what the tests do and do not reach, dead code included.

Coverage Investigation Methodology

Step 1: Collect Coverage Data

Use cargo-llvm-cov to generate line-level coverage. The tool must be installed via cargo install cargo-llvm-cov.

Each cargo llvm-cov that is not report runs the whole test suite again, so run the tests once and read every report from that run:

cargo llvm-cov clean --workspace   # a stale profile would merge into this run
cargo llvm-cov --no-report -p wado-compiler -p wado-compiler-tests
cargo llvm-cov report --html                                   # target/llvm-cov/html/
cargo llvm-cov report --json --output-path scratchpad/cov.json
jq '.data[0].totals.lines' scratchpad/cov.json                  # overall

Step 2: Identify Low-Coverage Files

Sort files by uncovered line count to find the biggest improvement opportunities:

jq -r '.data[0].files[] | "\(.summary.lines.count - .summary.lines.covered)\t\(.summary.lines.percent | round)%\t\(.filename)"' \
  scratchpad/cov.json | sort -rn | head -30

Focus on files with:

  • High absolute uncovered lines (biggest bang for buck)
  • Low percentage but meaningful code (not just error paths)

Step 3: Classify Coverage Gaps

For each low-coverage file, classify gaps into categories:

CategoryActionCoverage Impact
Dead codeDelete itImmediate improvement, no test needed
Untested language featuresAdd e2e test fixturesMedium effort, high impact
Error handling pathsAdd compile_error test fixturesLow effort
Optimizer-specific pathsAdd wir_expect/wir_not_expect testsMedium effort
Edge cases in codegenAdd targeted e2e testsVaries

Step 4: Analyze a Specific File

To see exactly which lines are uncovered in a file:

# lcov from the same run
cargo llvm-cov report --lcov --output-path scratchpad/lcov.info

# Or read the HTML report from Step 1:
# target/llvm-cov/html/src/synthesis/inspect.rs.html etc.

To find dead code (functions never called):

# Check if a function has any callers
rg 'function_name' wado-compiler/src/ --type rust

Step 5: Add E2E Tests

E2E tests are .wado files in wado-compiler/tests/fixtures/. Each file has a __DATA__ section with JSON expectations.

Guidelines for test files:

  • Name files based on language features, not compiler components (e.g., closure_nested.wado not codegen_closure.wado)
  • Prefer adding test cases to existing fixture files when the feature group matches
  • Each fixture group shares a filename prefix (e.g., closure_*, global_*, match_*)
  • After adding new files: touch wado-compiler/tests/e2e.rs to trigger rediscovery

Test patterns that maximize coverage:

  • test "..." { ... } with {"test": {}} — exercises codegen without needing stdout
  • compile_error tests — exercise error reporting paths
  • wir_expect:O2 / wir_not_expect:O2 — exercise optimizer paths
  • Template strings with {expr:?} — exercise inspect/display synthesis

Step 6: Verify Improvement

After changes, run Step 1 again and compare jq '.data[0].totals.lines.percent' scratchpad/cov.json with the earlier total.

Key Insights from Past Investigations

  • synthesis/inspect.rs (~1400 lines) has dead code for type variants that never appear in practice (e.g., Never type inspect). Deleting dead code directly improves coverage.
  • codegen.rs and emit.rs are the largest files but have relatively high coverage. Incremental improvements here require many targeted tests.
  • The compiler's test suite is driven by datatest_mini which discovers .wado fixtures at compile time — new files need touch wado-compiler/tests/e2e.rs.
  • wado test runs test-world fixtures; wado run runs CLI-world fixtures. Choose the right world for your test.
  • Coverage runs all optimization levels by default in CI; locally only O0 and O2 run unless WADO_FULL_TEST=1.
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall