arbiterForge/codeArbiter / plugins/ca/skills/tribunal
tribunal
VerifiedRun an opt-in deep codebase audit with persisted findings. Confirm cost before dispatch; filing and telemetry need separate approval.
references/lenses/test-fidelity.md
View on GitHub# test-fidelity — lens mandate
Executed by `tribunal-lens-reviewer` under the `test-fidelity` assignment.
## Purpose / failure family
Find doubles, fixtures and simulated boundaries that misrepresent the relied-on
contract and conceal real failures. Test-fidelity owns test-boundary fidelity.
## Applicability
Tests replace a producer, external system, clock, scheduler or lifecycle with a
model whose accuracy matters to their behavioral claim.
## Skip conditions
No relevant substituted boundary or fixtures. Do not require a real external
service merely to replace a faithful, bounded test double.
## Scope emphasis
Test seams and their actual producer/consumer contracts: serialized shapes,
variants, error behavior, side effects, ordering, cancellation and cleanup.
## Required reading
- Finding record (`${CLAUDE_PLUGIN_ROOT}/skills/tribunal/references/finding-record.md`) and review risk (`${CLAUDE_PLUGIN_ROOT}/skills/tribunal/references/review-risk.md`).
- Verification quality (`${CLAUDE_PLUGIN_ROOT}/skills/tribunal/references/verification-quality.md`) — shared oracle discipline and complementary ownership.
- `${CLAUDE_PROJECT_DIR}/.codearbiter/tech-stack.md` — mock/factory conventions and
test harnesses; the actual producer, consumer and relevant public contract.
## Deterministic probes
Locate substituted modules and fixtures; compare their fields, variants, error
shapes and timing assumptions with producer signatures, schemas and contract
tests. Inspect casts and TODOs as leads, never as defect classifiers.
## Review questions
- Can the real producer return a state or failure the double makes impossible?
- Does the consumer rely on ordering, cancellation, side effects or cleanup that
the fake silently omits?
- Does a stale fixture or cast bypass an invariant and let a real regression pass?
- Is any reduced fixture intentionally narrow enough for the tested obligation?
## False-positive guards / non-findings
Faithful mocks and purpose-built minimal fixtures are legitimate. A real producer
existing does not invalidate a double. Generated tests, casts and temporary
comments do not prove drift or explain who authored the fixture.
## Evidence requirements
Show the exact test/producer/consumer mismatch and the regression it hides.
For an absent contract check, state the search universe including alternate
harnesses and trace the owner; missing data irrelevant to the test is not drift.
## Exposure metric
Count of distinct substituted contracts examined, not mock-library call sites.
## Escalation / cross-lens handoff
Coverage owns missing behavioral cases and weak assertions without a seam
mismatch. Route product type-boundary defects to typesafety and runtime failures
to their domain lens; preserve one finding per root mechanism.
## Out of scope
Blanket replacement of mocks, author-intent inference, generic test counts and
implementation of new fixtures or tests.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |