nWave-ai/nWave / nWave/skills/nw-agent-testing
nw-agent-testing
Verified5-layer testing approach for agent validation including adversarial testing, security validation, and prompt injection resistance
Agent Testing Framework
5-Layer Testing Approach
Layer 1: Output Quality (Unit-Level)
Validate agent produces correct, well-structured outputs for typical inputs.
Test: Agent follows workflow phases | Outputs match expected format/structure | Domain-specific rules correctly applied | Token efficiency within bounds
How: Manual invocation with representative inputs. Check against acceptance criteria in agent description.
Layer 2: Integration / Handoff Validation
Validate correct input/output between agents in workflows.
Test: Input parsing handles upstream format | Output format matches downstream expectations | Error signals propagate correctly | Subagent mode activation works (skip greet, execute autonomously)
How: End-to-end workflow execution through full agent chain (e.g., DISCUSS -> DESIGN -> DELIVER).
Layer 3: Adversarial Output Validation
Challenge validity of agent outputs rather than accepting at face value.
Test: Source verification (cited sources real and accurate?) | Bias detection (favors one approach without evidence?) | Edge case coverage | Completeness (required sections present?)
How: Peer review by -reviewer agent using structured critique dimensions.
Layer 4: Adversarial Verification (Peer Review)
Independent review to catch biases and blind spots in agent design.
Test: Definition follows validation checklist? | Redundant Claude default instructions? | Over/under-specified? | Could simpler agent achieve same results?
How: @nw-agent-builder validates via nw-ab-validate-spec (the nw-ab-validation-checklist items) or @nw-agent-builder-reviewer runs structured review.
Layer 5: Security Validation
Test resilience against misuse and prompt injection.
Test: Tool restriction enforcement | maxTurns respected | Permission mode correctly scoped | Agent stays within declared scope
How: Frontmatter fields enforce at platform level. Verify configuration.
Prompt Injection Resistance
Claude Code platform provides injection resistance through: subagent isolation (own context, no sub-subagents) | Tool restriction via frontmatter tools | Permission modes via permissionMode | Hook-based validation (PreToolUse, PostToolUse)
Do NOT add prose-based injection defense. Configure platform features:
---
tools: Read, Glob, Grep # Only tools this agent needs
maxTurns: 40 # Minimum budget for a complete turn
permissionMode: default # User approves dangerous actions
---
Security Validation Checklist
-
toolsrestricted to minimum necessary (least privilege) -
maxTurnsset to prevent runaway execution -
permissionModeappropriate for risk level - No
Bashunless agent requires command execution - No
Writeunless agent creates/modifies files - Description accurately describes scope
- Subagent mode handles autonomous execution correctly
- No sensitive data hardcoded in definition
Testing Workflow for New Agents
- Create with minimal definition
- Layer 1: Invoke with 2-3 representative inputs, check outputs
- Layer 2: Run in workflow chain if applicable
- Fix failures observed
- Validate: Run
nw-ab-validate-spec - Iterate: Add instructions only for observed failure modes
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 466k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 466k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 466k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 192k | Updated | Claude CodeCodex | Verified | Install |