Basinfy/Basinfy-SaaS-Studio / .agents/skills/saas-smoke-check
saas-smoke-check
UnverifiedRun critical-path automated test suite under timeout as a pre-QA smoke gate.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 5.2 KB |
| references/1-detect-setup.md | markdown | 5.6 KB |
| references/2-run-tests.md | markdown | 1.7 KB |
| references/3-coverage.md | markdown | 1.6 KB |
| references/4-manual-checks.md | markdown | 5.7 KB |
| references/5-report.md | markdown | 6.7 KB |
| references/6-write-gate.md | markdown | 4.6 KB |
references/5-report.md
View on GitHub# /saas-smoke-check — Phase 5: Generate Report
> Part of `/saas-smoke-check`. `SKILL.md` names when to read this file; the settings it
> resolved and the rules under its later headings still apply here.
## Phase 5: Generate Report
**In this file:**
- The report template, with the verdict rules
- `NOT ASSESSED` and where it ranks
Assemble the full smoke check report:
````markdown
## Smoke Check Report
**Date**: [date]
**Sprint**: [sprint name / number, or "Not identified"]
**Engine**: [engine]
**QA Plan**: [path, or "Not found — run /saas-qa-plan first"]
**Argument**: [sprint | quick | blank]
---
### Automated Tests
**Status**: [PASS ([N] tests, [N] passing) | PASS WITH WARNINGS ([N] tests
passing; [the runner's warning, e.g. [N] orphan nodes leaked]) | FAIL ([N]
failures, or did not complete: [timeout | compile, parse or build error | no
results file]) | NOT ASSESSED ([reason, e.g. zero tests executed, runner could
not run]) | NOT RUN ([reason]) | WAIVED (qa.level: minimal — no integration tests)]
[FastAPI: one Status line per test suite — Unit and Integration.]
[WAIVED only:] **Build check**: [PASS — `[command]` exit 0 | FAIL — [reason] |
NOT RUN — [commands.smoke and commands.build unset | executable not found]]
[If FAIL, list failing tests:]
- `[test name]` — [brief failure description from runner output]
[If NOT RUN:]
"Manual confirmation required: did tests pass in your local IDE or CI? This
will determine whether the automated test row contributes to a FAIL verdict."
---
### Test Coverage
| Story | Type | Test File | Coverage Status |
|-------|------|-----------|----------------|
| [title] | Logic | `tests/unit/[system]/[slug]_test.[ext]` | COVERED |
| [title] | Visual/Feel | `production/qa/evidence/[slug]-screenshots.md` | MANUAL |
| [title] | Logic | — | MISSING ⚠ |
| [title] | Config/Data | — | EXPECTED |
**Summary**: [N] covered, [N] manual, [N] missing, [N] expected.
[If `quick`:] "Coverage: not checked (quick run) — run `/saas-smoke-check sprint`
before hand-off." in place of the table and summary.
---
### Manual Smoke Checks
- [x] Build launched this session — yes [or: `NOT RUN — build not launched this session`; the batches below were not asked]
- [x] Application boots without crash — PASS
- [x] New product starts — PASS
- [x] [Core workflow] — PASS
- [ ] [Other check] — FAIL: [user's description]
- [x] Database persistence / hydration — PASS
- [-] Performance — not checked this session
---
### Missing Test Evidence
Stories that must have test evidence before they can be marked COMPLETE via
`/saas-story-done`:
- **[story title]** (`[path]`) — Logic story has no test file.
Expected location: `tests/unit/[system]/[story-slug]_test.[ext]`
[If none:] "All Logic and Integration stories have test coverage."
[If `quick`:] "Not checked (quick run)." — never the line above, which a quick
run has not shown.
---
### Platform-Specific Results *(only if `--platform` was provided)*
| Platform | Checks Run | Passed | Failed | Platform Verdict |
|----------|-----------|--------|--------|-----------------|
| Desktop | [N] | [N] | [N] | PASS / FAIL |
| Tablet | [N] | [N] | [N] | PASS / FAIL |
| Mobile | [N] | [N] | [N] | PASS / FAIL |
**Platform notes**: [any platform-specific observations not captured in pass/fail]
Any platform with one or more FAIL checks contributes to the overall FAIL verdict.
---
### Verdict: [PASS | PASS WITH WARNINGS | NOT ASSESSED | FAIL]
[Verdict rules — first matching rule wins:]
**FAIL** if ANY of:
- Automated test suite ran and reported one or more test failures
- The automated suite did not complete: a timeout, a compile, parse or build
error, or no results file after the run (Phase 2 names each engine's form)
- The build check (WAIVED path) failed: a non-zero exit, a timeout, or a Next.js / TypeScript
build error or compilation failure
- Any Batch 1 (core stability) check returned FAIL
- Any Batch 2 (primary sprint feature or regression check) returned FAIL
**NOT ASSESSED** if ANY of:
- The automated suite is **unconfirmed NOT RUN** — nobody has reported a result
- The runner could not run (test runner binary missing or dependencies not installed), or it ran and executed zero tests
(`No test cases found`, `testcasecount="0"`, `No test suites matched`)
- E2E / Integration tests exist and are `E2E: NOT RUN`, unconfirmed — the
unit test result does not cover them
- A Batch 1 or Batch 2 check could not be executed (no build, environment not
configured, service unavailable) as opposed to executing and failing —
including a Batch 1 answer that the dev server was not launched this session
- **Any story's coverage row is `UNKNOWN`** (Phase 3: story file missing or
unreadable). A story nobody could read is not a story with no gaps — without
this line, a run where *every* row is UNKNOWN and the suite passes matches
**PASS**, because PASS only requires "no MISSING entries"
- **Batch 3 was offered and skipped** ("Performance not checked this session").
It is not a FAIL, not an execution failure, and not "PASS or N/A", so without
this line it matches no rule at all and renders as `[-]` beside a PASS
**PASS WITH WARNINGS** if ALL of:
- Automated tests PASS or PASS WITH WARNINGS, WAIVED at `qa.level: minimal`, or
NOT RUN **and the developer has confirmed the result from their local IDE or CI**
- All Batch 1 and Batch 2 smoke checks PASS
- At least one warning: a Logic/Integration story with MISSING test evidence,
a runner warning on a passing suite (e.g. unhandled promise rejections or memory leaks), or
a NOT RUN suite the developer confirmed
**PASS** if ALL of:
- Automated tests PASS, or WAIVED (`qa.level: minimal` and no integration tests exist —
the build check, when `commands.smoke` or `commands.build` is set, and the
launch and critical-path batches below then carry the verdict)
- All smoke checks in all batches PASS or N/A
- No MISSING test evidence entries
[On a `quick` run, any verdict carries the label: `PASS (quick — coverage and
Batch 3 not checked)`. A quick run can pass; it never reads as a full one.]
````
**`NOT ASSESSED` — the build nobody could check.** Rank: it **outranks PASS and
PASS WITH WARNINGS** and **ranks below FAIL**. A suite that never ran has not
shown the build is healthy; a suite that ran and failed is the more actionable
finding and must not be demoted behind one that did not run.
The rule below — **never treat NOT RUN as an automatic FAIL** — still holds:
NOT ASSESSED is not a FAIL, and it ranks below one. But an unrun suite never
resolves to a *pass* verdict while waiting for a confirmation that may never
come. Confirmed NOT RUN (the developer reports the
result from their own IDE or CI) still lands at PASS WITH WARNINGS, because
somebody did look.
---
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |