ash1794/vibe-engineering / plugins/vibe-engineering/skills/vibe-adversarial-test-generation
vibe-adversarial-test-generation
VerifiedGenerates edge case, failure mode, and spec-driven test cases. Covers boundary values, nil inputs, concurrency, resource exhaustion, malformed data, and requirement-linked traceability tests. Use after happy-path tests exist and before claiming coverage is complete, when requirements lack tests, or before a security review.
vibe-adversarial-test-generation
Happy-path tests prove your code works. Adversarial tests prove it doesn't break. Spec-linked tests prove it does what it's supposed to.
When to Use This Skill
- After writing happy-path tests for a feature
- Before claiming test coverage is complete
- When spec requirements lack corresponding tests
- When preparing for a security review
- When
vibe-spec-syncorvibe-coverage-enforcerreports uncovered requirements
When NOT to Use This Skill
- Before happy-path tests exist (write those first)
- For throwaway/prototype code
- When the function is trivially simple (e.g., getters/setters)
Modes
Mode 1: Adversarial (Edge Cases)
Generate tests that break assumptions across 6 categories.
1. Boundary Values
- Zero, one, max, max+1 for all numeric inputs
- Empty string, single char, max-length string
- Empty array, single element, very large array
- Exactly at threshold values
2. Nil/Null/Undefined
- nil pointer as receiver
- nil arguments to every parameter
- nil nested fields in structs
- Returning nil where non-nil expected
3. Type Edge Cases
- Unicode: emoji, RTL text, zero-width chars, combining chars
- Strings: newlines, tabs, null bytes, control characters
- Numbers: NaN, Infinity, -0, very large, very small
- Dates: leap year, DST transitions, timezone boundaries, epoch
4. Concurrency
- Two goroutines/threads calling the same function
- Read during write
- Close during use
- Cancel during operation
5. Resource Exhaustion
- Very large inputs (10MB string, 1M element slice)
- Disk full simulation
- Network timeout simulation
- Memory pressure
6. Malformed Input
- Invalid JSON/YAML/XML
- Truncated input (cut off mid-field)
- Wrong types (string where int expected)
- Extra fields, missing required fields
- SQL injection patterns, XSS payloads (for external inputs)
Mode 2: Spec-Driven (Requirement Coverage)
Generate tests that trace back to specific spec requirements.
-
Read the spec — Find the specification document for the feature under test
-
Extract requirements — Parse each requirement into an atomic, testable statement:
- "Users can reset passwords via email" → testable
- "The system should be fast" → not testable (flag it)
-
Map existing tests to requirements — Scan test files for:
- Comment markers:
# req:[REQ-ID]or// req:[REQ-ID] - Function name patterns:
test_req_[REQ-ID]_* - If no markers exist, use semantic matching (test name/body vs requirement text)
- Comment markers:
-
Identify uncovered requirements — Requirements with no mapped tests
-
Generate tests for uncovered requirements:
- One test function per requirement minimum
- Name format:
test_req_[REQ-ID]_[description](e.g.,test_req_AUTH003_password_reset_sends_email) - First line of test body MUST include traceability marker:
# req:AUTH-003// req:AUTH-003// req:AUTH-003 - Test must assert actual behavior against the requirement, not just call the function
- No
skip(), noTODO, no empty bodies
-
Report coverage delta:
- Requirements covered before: X/N
- Requirements covered after: Y/N
- Remaining uncovered (with reasons — e.g., "requires external service mock")
Steps (Adversarial Mode)
- Read the function under test — understand inputs, outputs, side effects
- For each input parameter, generate adversarial values from each category
- For each adversarial input, determine expected behavior:
- Should it return an error? (most common)
- Should it handle gracefully? (fallback behavior)
- Should it panic? (almost never the right answer)
- Write the tests as table-driven test cases
- Run them and fix any unexpected panics or wrong error handling
Steps (Spec-Driven Mode)
- Read the spec and extract atomic requirements with IDs
- Scan existing tests for requirement markers and semantic matches
- Generate a coverage map: requirement → test(s) or UNCOVERED
- Write tests for uncovered requirements with traceability markers
- Run tests and verify they pass against current implementation
- Report the before/after coverage delta
Output Format
Adversarial Tests: [Function Name]
Tests Generated: X Categories Covered: Y/6
| # | Category | Input | Expected | Actual |
|---|---|---|---|---|
| 1 | Nil input | nil | ErrNilInput | PASS |
| 2 | Empty string | "" | ErrEmpty | PANIC! |
Issues Found
- [Function] panics on nil input (should return error)
Spec-Driven Tests: [Feature/Spec Name]
Spec: [path/to/spec.md] Requirements Found: N Previously Covered: X/N (Y%) Now Covered: Z/N (W%)
| REQ ID | Requirement | Test Status | Test Function |
|---|---|---|---|
| AUTH-001 | Login requires email + password | Covered | test_req_AUTH001_login_requires_credentials |
| AUTH-002 | Failed login locks after 5 attempts | NEW | test_req_AUTH002_lockout_after_five_failures |
| AUTH-003 | Password reset sends email | UNCOVERED | (requires email service mock) |
Remaining Gaps
- AUTH-003: Requires email service mock — suggest adding mock in conftest.py
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 470k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 470k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 470k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |