Skip to content

borghei/Claude-Skills / project-management/execution/test-scenarios

test-scenarios

Verified

Writes QA cases from a feature spec across happy, edge, error, empty, concurrent, accessibility and security paths, then audits coverage. Use when planning QA for a feature, or finding coverage gaps after an incident.

Test Scenarios

The agent writes the scenarios from the spec; test_scenario_generator.py then audits the list by counting scenarios per category against what the feature type needs. The script generates nothing despite its name.

Done looks like: every user goal has a happy path; each input has boundary and weird-but-valid cases; each failure mode has a user-visible outcome; empty states, accessibility and security are covered for every UI feature; concurrency for anything shared or financial; localization when more than one locale ships; at least one P0; the audit exits 0.

When to Use

  • After PRD approval and before implementation, or when sizing test effort in sprint planning
  • Reviewing whether a test plan or a PR's tests cover enough
  • Bug-bash preparation, or after a production incident to find the missed category

Categories

Happy path, edge, error, empty state, concurrent, accessibility (a11y), security; plus performance, localization and cross-platform when they apply. Examples per category: references/scenario-categories.md.

Clarify First

Before generating scenarios, settle the inputs below. When one is missing or vague and would change the result, ask about it; when it wouldn't, use a sensible default and say which one you used.

  • Feature type: form, browse/list, real-time/collab, file upload, payment, or bulk operation (drives the scenario count per category in the coverage rubric).
  • Risk / sensitivity profile: auth, payment, or PII involved (scales up security and concurrency coverage in the risk-weighted selection).
  • The spec's inputs, outputs, and side effects: what users provide, expect, and what changes in the system (drives edge cases in Step 3 and error handling in Step 4).

Ask about the two or three inputs that matter most for this request, not the whole list. If the user wants a result straight away, produce it and state your assumptions at the top.

Quick Start

# Signup form with full coverage: exit 0
python3 project-management/execution/test-scenarios/scripts/test_scenario_generator.py \
  --input project-management/execution/test-scenarios/assets/sample_scenarios.json --format markdown
# Payment step with no error or security scenarios and a mistyped category: exit 1
python3 project-management/execution/test-scenarios/scripts/test_scenario_generator.py \
  --input project-management/execution/test-scenarios/assets/sample_scenarios_flawed.json --format markdown
# Input schema and flags
python3 project-management/execution/test-scenarios/scripts/test_scenario_generator.py --help

Tools Overview

test_scenario_generator.py is the quality gate for this skill. What the audit checks: scenario counts per category against the feature type's recommended range (fail when happy path, edge, error or security has none; warn when below range; info when above); unknown category names; concurrency for collaborative features; localization when several locales are supported; cross-platform coverage; at least one P0. It counts; it does not judge whether a scenario is any good.

Exit codes: 0 no fail findings · 1 at least one fail (a core category with zero scenarios) · 2 unreadable file, invalid JSON or a top level that is not an object. JSON by default, --format markdown, --output FILE.

Workflow

  1. Read the spec for user goals, inputs, outputs and side effects.
  2. Write scenarios per category. You know the standard techniques (boundaries, wrong types, 4xx/5xx/timeouts, keyboard and screen reader, authorisation and injection); spend the effort on what is specific to this feature: its side effects, its shared state, its money or PII. Payment and collaboration features need concurrency and idempotency cases; uploads need file-abuse cases.
  3. Weight by risk. Always cover happy paths and empty states; cover edge cases by likelihood x severity; scale security with sensitivity. Mark P0 the scenarios whose failure would stop the release.
  4. Audit. Write the list as JSON (schema in --help; assets/sample_scenarios.json) and run the audit. It must pass with no fail findings; then verify that each under-covered category is either filled or consciously out of scope. Recommended counts per feature type are in the appendix of references/scenario-categories.md.
  5. Hand to QA with the automated/manual split from the same appendix.

Anti-Patterns

Only happy paths

Mistake: The plan tests that the feature works when everything goes right. Why it happens: Happy paths come straight from the spec; failures have to be imagined. Instead: Write at least the recommended error and edge counts for the feature type before anything else is added.

"QA will figure it out"

Mistake: Scenarios are left to QA after the build. Why it happens: Scenario writing feels like QA's job. Instead: Write them before implementation; QA tests what is specified, and engineers build for cases they can see.

No concurrency for shared or financial state

Mistake: A payment or collaborative feature ships with no race or idempotency scenarios. Why it happens: Concurrency bugs do not show up in single-user demos. Instead: Cover double-submit, simultaneous edit and retry-after-timeout explicitly.

More gaps with fixes: references/coverage-anti-patterns.md.

Scope & Limitations

In Scope:

  • Test scenarios across seven categories: happy path, edge, error, empty state, concurrent, accessibility and security, plus performance, localization and cross-platform where they apply
  • Scenario counts per category by feature type: form, browse, real-time, upload, payment and bulk (table in the reference appendix, matching the script)
  • Risk-weighted selection and a default split between manual and automated testing per scenario type
  • A coverage audit that flags under-covered categories for the feature type (test_scenario_generator.py)
  • Test-plan audits and post-incident scenario gap analysis

Out of Scope:

  • Writing the PRD or acceptance criteria the scenarios come from (see project-management/execution/create-prd/ and project-management/execution/wwas/)
  • Implementing automated tests (see engineering/senior-qa/ and engineering/playwright-pro/)
  • A full WCAG accessibility audit (see engineering/a11y-audit/)
  • Security testing and penetration work (see engineering/security-audit/)

Important Caveats:

  • Despite its name, test_scenario_generator.py does not write scenarios. It audits a hand-written JSON list and counts scenarios per category.
  • The tool checks coverage by count, not scenario quality. A category can pass with weak scenarios.
  • The per-category counts are starting defaults. The team adjusts them for the risk profile of the feature.
  • The team decides which scenarios to automate and which edge cases matter enough to test.

Files

FilePurpose
scripts/test_scenario_generator.pyCoverage audit by category (it does not generate scenarios)
assets/sample_scenarios.jsonSignup form, 19 scenarios; exits 0
assets/sample_scenarios_flawed.jsonUnder-covered payment step; exits 1
assets/scenario_template.mdScenario template

Reference Documentation

Related skills

  • project-management/execution/create-prd — upstream spec
  • project-management/execution/wwas — acceptance criteria
  • engineering/senior-qa — implementation
  • engineering/code-reviewer — review coverage
  • product-team/spec-to-repo — translating spec to tickets
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall