Skip to content

Basinfy/Basinfy-SaaS-Studio / .agents/skills/saas-smoke-check

saas-smoke-check

Unverified

Run critical-path automated test suite under timeout as a pre-QA smoke gate.

FileKindSize
SKILL.mdskill md5.2 KB
references/1-detect-setup.mdmarkdown5.6 KB
references/2-run-tests.mdmarkdown1.7 KB
references/3-coverage.mdmarkdown1.6 KB
references/4-manual-checks.mdmarkdown5.7 KB
references/5-report.mdmarkdown6.7 KB
references/6-write-gate.mdmarkdown4.6 KB

references/5-report.md

View on GitHub
# /saas-smoke-check — Phase 5: Generate Report

> Part of `/saas-smoke-check`. `SKILL.md` names when to read this file; the settings it
> resolved and the rules under its later headings still apply here.

## Phase 5: Generate Report

**In this file:**

- The report template, with the verdict rules
- `NOT ASSESSED` and where it ranks

Assemble the full smoke check report:

````markdown
## Smoke Check Report
**Date**: [date]
**Sprint**: [sprint name / number, or "Not identified"]
**Engine**: [engine]
**QA Plan**: [path, or "Not found — run /saas-qa-plan first"]
**Argument**: [sprint | quick | blank]

---

### Automated Tests

**Status**: [PASS ([N] tests, [N] passing) | PASS WITH WARNINGS ([N] tests
passing; [the runner's warning, e.g. [N] orphan nodes leaked]) | FAIL ([N]
failures, or did not complete: [timeout | compile, parse or build error | no
results file]) | NOT ASSESSED ([reason, e.g. zero tests executed, runner could
not run]) | NOT RUN ([reason]) | WAIVED (qa.level: minimal — no integration tests)]

[FastAPI: one Status line per test suite — Unit and Integration.]

[WAIVED only:] **Build check**: [PASS — `[command]` exit 0 | FAIL — [reason] |
NOT RUN — [commands.smoke and commands.build unset | executable not found]]

[If FAIL, list failing tests:]
- `[test name]` — [brief failure description from runner output]

[If NOT RUN:]
"Manual confirmation required: did tests pass in your local IDE or CI? This
will determine whether the automated test row contributes to a FAIL verdict."

---

### Test Coverage

| Story | Type | Test File | Coverage Status |
|-------|------|-----------|----------------|
| [title] | Logic | `tests/unit/[system]/[slug]_test.[ext]` | COVERED |
| [title] | Visual/Feel | `production/qa/evidence/[slug]-screenshots.md` | MANUAL |
| [title] | Logic | — | MISSING ⚠ |
| [title] | Config/Data | — | EXPECTED |

**Summary**: [N] covered, [N] manual, [N] missing, [N] expected.

[If `quick`:] "Coverage: not checked (quick run) — run `/saas-smoke-check sprint`
before hand-off." in place of the table and summary.

---

### Manual Smoke Checks

- [x] Build launched this session — yes [or: `NOT RUN — build not launched this session`; the batches below were not asked]
- [x] Application boots without crash — PASS
- [x] New product starts — PASS
- [x] [Core workflow] — PASS
- [ ] [Other check] — FAIL: [user's description]
- [x] Database persistence / hydration — PASS
- [-] Performance — not checked this session

---

### Missing Test Evidence

Stories that must have test evidence before they can be marked COMPLETE via
`/saas-story-done`:

- **[story title]** (`[path]`) — Logic story has no test file.
  Expected location: `tests/unit/[system]/[story-slug]_test.[ext]`

[If none:] "All Logic and Integration stories have test coverage."
[If `quick`:] "Not checked (quick run)." — never the line above, which a quick
run has not shown.

---

### Platform-Specific Results *(only if `--platform` was provided)*

| Platform | Checks Run | Passed | Failed | Platform Verdict |
|----------|-----------|--------|--------|-----------------|
| Desktop | [N] | [N] | [N] | PASS / FAIL |
| Tablet | [N] | [N] | [N] | PASS / FAIL |
| Mobile | [N] | [N] | [N] | PASS / FAIL |

**Platform notes**: [any platform-specific observations not captured in pass/fail]

Any platform with one or more FAIL checks contributes to the overall FAIL verdict.

---

### Verdict: [PASS | PASS WITH WARNINGS | NOT ASSESSED | FAIL]

[Verdict rules — first matching rule wins:]

**FAIL** if ANY of:
- Automated test suite ran and reported one or more test failures
- The automated suite did not complete: a timeout, a compile, parse or build
  error, or no results file after the run (Phase 2 names each engine's form)
- The build check (WAIVED path) failed: a non-zero exit, a timeout, or a Next.js / TypeScript
  build error or compilation failure
- Any Batch 1 (core stability) check returned FAIL
- Any Batch 2 (primary sprint feature or regression check) returned FAIL

**NOT ASSESSED** if ANY of:
- The automated suite is **unconfirmed NOT RUN** — nobody has reported a result
- The runner could not run (test runner binary missing or dependencies not installed), or it ran and executed zero tests
  (`No test cases found`, `testcasecount="0"`, `No test suites matched`)
- E2E / Integration tests exist and are `E2E: NOT RUN`, unconfirmed — the
  unit test result does not cover them
- A Batch 1 or Batch 2 check could not be executed (no build, environment not
  configured, service unavailable) as opposed to executing and failing —
  including a Batch 1 answer that the dev server was not launched this session
- **Any story's coverage row is `UNKNOWN`** (Phase 3: story file missing or
  unreadable). A story nobody could read is not a story with no gaps — without
  this line, a run where *every* row is UNKNOWN and the suite passes matches
  **PASS**, because PASS only requires "no MISSING entries"
- **Batch 3 was offered and skipped** ("Performance not checked this session").
  It is not a FAIL, not an execution failure, and not "PASS or N/A", so without
  this line it matches no rule at all and renders as `[-]` beside a PASS

**PASS WITH WARNINGS** if ALL of:
- Automated tests PASS or PASS WITH WARNINGS, WAIVED at `qa.level: minimal`, or
  NOT RUN **and the developer has confirmed the result from their local IDE or CI**
- All Batch 1 and Batch 2 smoke checks PASS
- At least one warning: a Logic/Integration story with MISSING test evidence,
  a runner warning on a passing suite (e.g. unhandled promise rejections or memory leaks), or
  a NOT RUN suite the developer confirmed

**PASS** if ALL of:
- Automated tests PASS, or WAIVED (`qa.level: minimal` and no integration tests exist —
  the build check, when `commands.smoke` or `commands.build` is set, and the
  launch and critical-path batches below then carry the verdict)
- All smoke checks in all batches PASS or N/A
- No MISSING test evidence entries

[On a `quick` run, any verdict carries the label: `PASS (quick — coverage and
Batch 3 not checked)`. A quick run can pass; it never reads as a full one.]
````

**`NOT ASSESSED` — the build nobody could check.** Rank: it **outranks PASS and
PASS WITH WARNINGS** and **ranks below FAIL**. A suite that never ran has not
shown the build is healthy; a suite that ran and failed is the more actionable
finding and must not be demoted behind one that did not run.

The rule below — **never treat NOT RUN as an automatic FAIL** — still holds:
NOT ASSESSED is not a FAIL, and it ranks below one. But an unrun suite never
resolves to a *pass* verdict while waiting for a confirmation that may never
come. Confirmed NOT RUN (the developer reports the
result from their own IDE or CI) still lands at PASS WITH WARNINGS, because
somebody did look.

---
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall