notque/vexjoy-agent / skills/process/testing
testing
VerifiedTesting: TDD, E2E, preferred patterns, test-value audits, verification, agent testing.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 23.1 KB |
| references/agents-examples-and-errors.md | markdown | 5.4 KB |
| references/agents-testing-patterns.md | markdown | 4.0 KB |
| references/audit-test-value.md | markdown | 8.3 KB |
| references/e2e-async.md | markdown | 8.9 KB |
| references/e2e-auth.md | markdown | 11.6 KB |
| references/e2e-financial-flows.md | markdown | 1.7 KB |
| references/e2e-playwright-patterns.md | markdown | 8.4 KB |
| references/e2e-templates.md | markdown | 6.9 KB |
| references/e2e-wallet-testing.md | markdown | 2.2 KB |
| references/patterns-blind-spot-taxonomy.md | markdown | 4.5 KB |
| references/patterns-fix-strategies.md | markdown | 5.0 KB |
| references/patterns-load-test-scenarios.md | markdown | 4.8 KB |
| references/patterns-preferred-pattern-catalog.md | markdown | 14.3 KB |
| references/patterns-quality-catalog.md | markdown | 8.5 KB |
| references/tdd-examples.md | markdown | 19.4 KB |
| references/tdd-phase-guidance.md | markdown | 10.1 KB |
| references/verify-adversarial-methodology.md | markdown | 9.5 KB |
| references/verify-checklist.md | markdown | 15.4 KB |
| references/verify-verification-examples.md | markdown | 14.6 KB |
references/agents-testing-patterns.md
View on GitHub# Testing Patterns Reference
Detailed patterns for agent testing using subagent dispatch.
## Pattern 1: Dispatch and Capture
Basic pattern for running a single test case:
```
# 1. Dispatch agent with test input
Task(
prompt="""
Review this Python function for issues:
```python
def get_user(id):
return db.execute(f"SELECT * FROM users WHERE id = {id}")
```
""",
subagent_type="reviewer-system"
)
# 2. Capture output verbatim
# 3. Compare to expected output
Expected: CRITICAL SQL injection finding
Actual: {what agent actually returned}
```
## Pattern 2: Negative Testing
Verify agent handles invalid inputs correctly:
```
# Test with missing required context
Task(
prompt="""
Review this code.
""",
subagent_type="reviewer-system"
)
# Expected: Agent should request more context or handle gracefully
# NOT: Agent should hallucinate code to review
```
## Pattern 3: Consistency Testing
Verify agent produces consistent outputs:
```
# Run same input 3 times
for i in 1..3:
Task(prompt=same_input, subagent_type=agent)
capture output[i]
# Compare outputs
# Structure should be identical
# Key findings should match
# Minor phrasing variation acceptable
```
## Pattern 4: A/B Comparison Testing
Compare agent variants:
```
# Use agent-comparison skill for formal A/B testing
# Or simple side-by-side:
Task(prompt=test_input, subagent_type="agent-v1")
capture output_v1
Task(prompt=test_input, subagent_type="agent-v2")
capture output_v2
# Compare quality, structure, correctness
```
## Pattern 5: Routing Verification
Verify correct agent is selected:
```
# Test routing logic by examining which agent handles request
# Check routing metadata matches behavior
# Example: "Review this Go code" should route to Go expert
# Example: "Check security of this API" should route to security reviewer
```
## Test Scenarios
### Scenario: Testing New Agents
Verify a new agent produces correct outputs for its intended purpose.
| Category | Purpose | Example |
|----------|---------|---------|
| Happy Path | Agent handles ideal input correctly | Valid code for code reviewer |
| Error Cases | Agent handles invalid input gracefully | Malformed input, missing context |
| Edge Cases | Agent handles boundary conditions | Empty input, very large input |
| Output Schema | Agent produces expected structure | Required sections present |
### Scenario: Testing Skill Invocation
- Skill triggers on expected phrases
- Skill produces expected output format
- Skill handles errors appropriately
- Skill follows its documented workflow
### Scenario: Testing Agent Interactions
- Correct agent selected for request
- Agent handoffs work correctly
- No conflicts between parallel agents
- Aggregated results are consistent
### Scenario: Testing Error Handling
- Invalid input handling
- Missing context handling
- Tool failures (simulated)
- Timeout behavior
### Scenario: Testing Output Format
- Required sections present
- Correct markdown formatting
- Expected fields populated
- No placeholder text
## Minimum Test Cases by Agent Type
| Agent Type | Minimum Tests | Required Coverage |
|------------|---------------|-------------------|
| Reviewer agents | 6 | 2 real issues, 2 clean, 1 edge, 1 ambiguous |
| Implementation agents | 5 | 2 typical, 1 complex, 1 minimal, 1 error |
| Analysis agents | 4 | 2 standard, 1 edge, 1 malformed |
## Test Report Template
```markdown
# Agent Test Report: {agent-name}
**Date:** {date}
**Version Tested:** {agent version}
**Tester:** Claude Code (testing-agents-with-subagents skill)
## Summary
| Metric | Result |
|--------|--------|
| Test Cases Run | N |
| Passed | N |
| Failed | N |
| Pass Rate | N% |
## Test Results
### T1: {test name}
- Status: PASS/FAIL
- Input: {input}
- Expected: {expected}
- Actual: {actual}
- Notes: {any observations}
[Continue for all tests]
## Issues Found
1. {Issue description} - Severity: HIGH/MEDIUM/LOW
## Recommendations
1. {Specific fix needed}
## Verdict
READY FOR DEPLOYMENT / NEEDS FIXES / REQUIRES REVIEW
```
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 468k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 468k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 468k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |