Skip to content

heymegabyte/agent-skills / skills/tdd-and-verification

tdd-and-verification

Verified

Canonical Test-Driven Development (Red-Green-Refactor), legacy characterization-first retrofit, Nx selective impact execution, 300-second CI gate, Cloudflare Browser Run parallel CDP orchestration, and 15-family (450-item) verification policy.

TDD & Verification Discipline (F01.01–F15.30)

This skill governs test selection, Red-Green-Refactor execution, legacy characterization, selective CI within a 300-second budget, parallel remote Chromium verification via Cloudflare Browser Run, and honest completion reporting across all 15 testing families (F01–F15, 450 obligations).

1. Core Operating Rules (Non-Negotiable)

  1. Behavioral Oracle First: Every test must assert observable domain outcomes, persisted state transitions, HTTP/RPC contracts, accessibility semantics, or security boundaries—never private implementation details or mock call counts alone (F01.01, F14.07).
  2. Genuine Red $\to$ Green $\to$ Refactor:
    • New features & bug fixes: Write the smallest automated test at the lowest convincing layer first. Run it and confirm it fails for the expected behavioral reason (not syntax, import, or harness errors). Write the minimal production code to pass, re-run, and refactor while keeping tests green (F14.01–F14.03).
    • Existing untested legacy code: Never delete working code merely to manufacture a fake test-first history. Write characterization tests that lock in current observable behavior first, verify they pass, then add a failing test for the new behavior or refactor safely (F01.30, F14.04).
  3. Critical-Path Retrofit Priority: Before expanding breadth, protect critical existing paths:
    • Authentication, session lifecycle, OAuth PKCE, Magic Links, and CSRF/Origin checks (F08.01–F08.08).
    • Multi-tenant and 6-role RBAC authorization (owner, publisher, editor, author, reviewer, viewer), including negative tests proving unauthorized mutations fail closed (403) (F02.07, F08.05).
    • Durable state transitions, immutable revision pointers, optimistic concurrency (baseRevisionId), and automatic rollback on post-publish smoke failure (F03.04, F09.01).
    • Consequential external side effects, idempotency keys, outbox/webhook replay safety, America/New_York DST slot scheduling, and integer micro-USD budget caps (F02.18, F05.03, F10.21).
    • Shared UI tokens, 15 OKLCH themes, WCAG 2.2 AA contrast, Storybook component states, and code-split Plate/Slate authoring (F02.01, F06.01).

2. Layer Selection Decision Matrix

Always choose the lowest layer that provides convincing fidelity, paired with a small set of real-browser critical journeys:

Change TypePrimary LayerSecondary / Sentinel LayerCacheable in CI?
Pure domain rule, parser, validator, cost math, DST slot calculationunit / property (Vitest)Contract schema checkYes (deterministic inputs)
Shared UI component, Plate/Slate block, theme token, keyboard/focus statecomponent_storybook + accessibilityAffected Storybook + Showcase E2EBuild/unit yes; live browser no
Worker route, API handler, auth/RBAC guard, DataForSEO client, SSRF guardworker_integration + contract + securityCritical browser smokeUnit/contract yes; live probe no
Queue consumer, cron scheduler, retry/backoff, checkpoint recoveryresilience_chaos + unit (injected clock)Idempotency replay testYes
User-visible route, checkout/publish flow, preview deploymente2e_browser (Cloudflare Browser Run / Playwright)Post-deploy console/CSP/axe gateNever (fresh execution required)

3. Nx Selective Execution & Uncertainty Fallbacks (@theme-tools/testing/nx-selector)

Use computeAffectedSelectionManifest (packages/testing/src/nx-selector.ts) and nx.json to select tasks within the 300-second CI budget:

  • Deterministic cacheable tasks (validate, typecheck, test, build): May reuse verified artifact/task cache when namedInputs (production, default, testingDiscipline, sharedGlobals) match.
  • Live browser specs (test:e2e, e2e:prod): Marked cacheable: false and freshExecutionRequired: true.
  • Mandatory Conservative Fallbacks: Expand automatically to the full deterministic suite + all critical browser sentinels whenever:
    1. baselineValidInGitHistory === false or the git base SHA is missing/shallow (F15.14).
    2. impactMapFresh === false or dynamic imports/routing changes have unmapped blast radius (F15.13).
    3. Global harness files (pnpm-lock.yaml, tsconfig.base.json, nx.json, playwright.config.ts, .github/workflows/, packages/testing/) are modified.

4. Five-Minute (300-Second) Required CI Gate (@theme-tools/testing/ci-gate)

The required CI path (.github/workflows/ci-required-5min.yml) enforces a strict 300-second wall-clock budget:

  • 0–30s (selection_and_restore): Checkout, pnpm store restore, baseline verification, computeAffectedSelectionManifest, and duration-balanced shard partitioning (partitionSelectedSpecsAcrossShards).
  • 30–120s (deterministic_and_build): Parallel static/typecheck, unit/property/contract suites, 15-theme contrast validation, and static preview builds (apps/web, apps/ui).
  • 120–270s (remote_browser_and_smoke): Sharded Playwright E2E journeys over Cloudflare Browser Run CDP against the verified preview artifact.
  • 270–300s (reconciliation_and_teardown): reconcileDeploymentGate audits every expected task and shard, verifies discoveredTestCount > 0, checks QuarantineRegistry (rejecting expired or unreplaced critical-path quarantines), and guarantees remote browser session teardown.

Fail-Closed Gate Rules:

  • Any missing, cancelled, timed-out, or zero-test shard/task fails the gate immediately (F12.11, F14.19).
  • Passing on retry is classified as flaky_passed_on_retry with the first-failure message/trace preserved (F13.06).

5. Cloudflare Browser Run Parallel CDP Rules (@theme-tools/testing/browser-run)

  1. Header Authentication & Secret Redaction: Connect via buildBrowserRunCdpConfig with Authorization: Bearer <token> in headers—never in query strings. Scrub tokens from logs, traces, and screenshots via redactBrowserRunDiagnostics.
  2. Rate & Concurrency Governance: BrowserRunPoolCoordinator enforces an account concurrency semaphore (default 16, max 200 on Workers Paid) and a sliding-window launch rate limiter (<= 3 new sessions/sec default, max 30/sec), respecting HTTP 429 Retry-After within runDeadlineMs.
  3. Worker Reuse & Context Isolation: Reuse one remote Chromium session per shard worker and allocate a fresh BrowserContext per test (allocateIsolatedContext), closing all sessions in finally (withWorkerSession).
  4. Preview Identity Verification: Call verifyPreviewDeploymentIdentity before running E2E assertions to confirm x-deployment-id and x-commit-sha match the tested revision; reject unexposed localhost URLs in remote Browser Run.

6. Anti-Gaming & Honest Verification Reporting (@theme-tools/testing/skill-eval-harness)

Before claiming completion, run detectAssertionWeakeningOrFalseGreen and report:

  • Exact commands executed, exit codes, durations, and test counts by layer.
  • Explicit separation between freshly_executed, reused_cached_evidence, and flaky_passed_on_retry.
  • Explicit disclosure of any skipped targets, not_applicable rationales, or missing external credentials. Never claim a live provider or remote browser ran when only local/cached fixtures executed.

7. Referenced 15 Testing Family Guides (F01–F15, 450 Recommendations)

Load only the family reference(s) matching your current task:

finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall