Skip to content

armelhbobdad/oh-my-skills / .claude/skills/skf-create-stack-skill

skf-create-stack-skill

Verified

Consolidated project stack skill with integration patterns — code-mode (analyzes manifests) or compose-mode (synthesizes from existing skills + architecture doc). Use when the user requests to "create a stack skill", "forge a stack", or "stack this project".

references/validate.md

View on GitHub
---
nextStepFile: 'report.md'
outputValidatorProbeOrder:
  - '{project-root}/_bmad/skf/shared/scripts/skf-validate-output.py'
  - '{project-root}/src/shared/scripts/skf-validate-output.py'
hashContentProbeOrder:
  - '{project-root}/_bmad/skf/shared/scripts/skf-hash-content.py'
  - '{project-root}/src/shared/scripts/skf-hash-content.py'
securityScanProbeOrder:
  - '{project-root}/_bmad/skf/shared/scripts/skf-security-scan.py'
  - '{project-root}/src/shared/scripts/skf-security-scan.py'
---

<!-- Config: communicate in {communication_language}. Artifact text in {document_output_language}. -->

# Step 8: Validate Output

## STEP GOAL:

Validate all written output files against their expected structure and verify confidence tier label completeness.

## Rules

- Validate structure and completeness, not content quality
- Only §3 writes to the committed package: `skill-check --fix` may rewrite `SKILL.md`, and a `body.max_lines` finding moves sections into `references/` (or runs `split-body --write`). Every other section leaves the package as step 7 committed it
- Advisory mode: always proceed to report regardless of findings

## MANDATORY SEQUENCE

### 1. Verify File Existence

Run the shared deterministic output validator once against the committed package — it checks the three core deliverable files' existence, SKILL.md frontmatter, and context-snippet format/token in a single call, so this step consumes its JSON rather than re-deriving those by hand. Resolve `{outputValidator}` from `{outputValidatorProbeOrder}` (first existing path wins). If neither candidate exists, log a WARNING (`"output validator unavailable — skf-validate-output.py missing"`) and fall back to the manual file/frontmatter/snippet checks below.

```bash
python3 {outputValidator} {skill_package} --generated-by create-stack-skill --skill-type stack --forge-tier {forge_tier}
```

Consume from its JSON:

- `files_found` — existence of `SKILL.md` / `context-snippet.md` / `metadata.json` (the three core deliverable rows below).
- `validation.skill_md.frontmatter` — feeds the frontmatter check in §3 (manual-fallback path).
- `validation.context_snippet.issues` — feeds §8.
- `validation.stack_counts` — the stack count-equalities, derived deterministically from disk + metadata by the `--skill-type stack` pass: `issues[]` (each `{severity, field, message}`, one per mismatch of `library_count` / `integration_count` / `confidence_distribution`; empty when all agree) and `observed` (`library_count_meta`, `ref_file_count`, `integration_count_meta`, `pair_file_count`, `confidence_sum`). Feeds the count rows in §5 and the confidence-sum row in §7 — do not re-count files or re-sum the distribution by hand.
- `validation.stack_structure`: the stack structure pass. The `check` of each `issues[]` entry (`{severity, check, field, message}`) names the section below that records it: `skill_md` (§4), `metadata` (§5), `references` (§6) or `tier_labels` (§7). Do not re-read the output files to check what it checks.

Under `--skill-type stack` the validator skips the individual-skill body and metadata passes (their `{"skipped": ...}` markers are no finding) and runs these two instead. When §3 changes the package it runs this command again, and §4 to §8 read the latest run. When the validator is unavailable, §4 to §7 each record one **WARNING** finding that their checks did not run.

Then confirm the remaining files the validator does not cover:

**Deliverables** (`{skill_package}`):
- [ ] SKILL.md · context-snippet.md · metadata.json — from `files_found` above
- [ ] references/ directory with per-library files
- [ ] references/integrations/ directory with pair files (if integrations detected)

**Workspace** (`{forge_version}`):
- [ ] provenance-map.json
- [ ] evidence-report.md

**Symlink:**
- [ ] `{skill_group}/active` exists and resolves to `{version}`

Record any missing files (from `files_found` or the manual rows) as **ERROR** findings.

### 2. Check Tool Availability

Probe skill-check with `--no-install` to avoid cold-install hangs, wrap in a short timeout, and treat any hang or non-zero exit as unavailable (S14):

```bash
timeout 10s npx --no-install skill-check -h
```

- If exits 0: Use skill-check for automated validation in section 3.
- If exits non-zero, times out, or returns "command not found": Use manual fallback paths, append a `workflow_warnings[]` entry (`step: "step-08"`, `severity: "warn"`, `code: "skill-check-unavailable"`, `message: "skill-check unavailable: manual fallback checks used"`) so step 9 does not report the run as skill-check validated, recording it at once (write its `[step-08/warn] skill-check-unavailable: {message}` line to `{run_dir}/warning.txt` with a file write, then run `uv run {emitEnvelopeHelper} record --run-dir "{run_dir}" --warning "$(cat "{run_dir}/warning.txt")"`), and list every skipped check in the §10 validation results.

**Important:** Do not assume availability — empirical check required.

### 3. Validate SKILL.md via skill-check (if available)

**If available**, resolve `{hashContentHelper}` ← first existing path in `{hashContentProbeOrder}`, then run in order: `uv run {hashContentHelper} hash <skill-dir>/SKILL.md`, `npx skill-check check <skill-dir> --fix --format json --no-security-scan`, then `uv run {hashContentHelper} hash <skill-dir>/SKILL.md` again, keeping the `content_hash` each `hash` call prints.

This validates frontmatter, description, body limits, links and formatting, and auto-fixes deterministic issues. Parse JSON for `scores[].score` (match the entry by `relativePath`/`skillId`; falls back to a top-level `qualityScore` on older skill-check builds) and `diagnostics[]`. skill-check's JSON lists no fix: SKILL.md changed across the `--fix` call when the two `content_hash` values differ, and is taken as changed, not hashed, when no `{hashContentProbeOrder}` path exists (run the skill-check call alone) or a `hash` call prints no `content_hash`.

Bind `{quality_score}` ← that score, from the latest skill-check run (§3 runs it again after a split); `{quality_score}` is null when skill-check is unavailable. It is the skill-check score, not a test-skill score, and step 9 puts it in the result envelope and the result contract.

**Post-fix provenance drift guard (S15):** When SKILL.md changed across the `--fix` call, `skill-check --fix` modified it after step 7 wrote it, so the `metadata.json` hashes and provenance recorded against the earlier body may now be stale. Emit one **WARNING** finding, `"skill-check --fix modified SKILL.md after step 7 wrote it: metadata.json hashes and provenance may be out of date"` (`"SKILL.md may have been rewritten (not hashed) by skill-check --fix after step 7 wrote it: metadata.json hashes and provenance may be out of date"` when SKILL.md was not hashed), rather than silently accepting the fix, so the drift is surfaced. If the caller wants authoritative metadata, they should re-run the workflow.

**If `body.max_lines` reported**, prefer selective split: extract only the largest Tier 2 section(s) to `references/`, keeping Tier 1 content inline (inline passive context achieves 100% task accuracy vs 79% for on-demand retrieval). For a stack capstone the canonical split is the catalog (`Library Reference Index` + `Per-Library Summaries`) → `references/stack-catalog.md`, leaving an inline pointer (see `{stackSkillTemplatePath}` "Sizing Guidance"). The moved catalog's links resolve from `references/`, so rewrite each `[ref](references/{name}.md)` in it to `[ref]({name}.md)`. This is the **intended** large-stack layout, not a violation: §4 below accepts the pointer form, so clearing the skill-check body ERROR this way does not also trip the structure check. Fall back to `npx skill-check split-body <skill-dir> --write` if not feasible. After the split, verify that any in-SKILL.md anchor links (e.g. to the catalog/pointer or other moved sections) still resolve, and that every link inside a moved section resolves from its new file.

**Re-validate a changed package.** When SKILL.md changed across the `--fix` call (not hashed included) or a split ran, run the §1 `{outputValidator}` command again (after a split, skill-check too), so §4 to §8 record the package as it now stands.

**If unavailable**, do not hand-walk the frontmatter: use `validation.skill_md.frontmatter` from the §1 output-validator run, which checks delimiters, `name` format + directory match (`{stack_name}`), `description` presence/length, and unknown fields against the agentskills.io allow-set. Record each reported issue at its severity as a **WARNING** finding. (If the output validator was *also* unavailable in §1, fall back to the manual checklist: `---` delimiters; `name` lowercase-alphanumeric-plus-hyphens 1-64 chars matching `{stack_name}`; `description` present and 1-1024 chars; only `name`/`description`/`license`/`compatibility`/`metadata`/`allowed-tools` permitted.) Invalid frontmatter will fail `npx skills add` and `npx skill-check check`.

### 4. Validate SKILL.md Body Structure

Record each `validation.stack_structure.issues[]` entry whose `check` is `skill_md` as a **WARNING** finding. The catalog's pointer form (a `## Library Catalog` section linking `references/stack-catalog.md`) is the intended large-stack layout (§3), not a violation.

### 5. Validate metadata.json Fields

Record each `validation.stack_structure.issues[]` entry whose `check` is `metadata` as a **WARNING** finding.

**Count equalities (library / integration):** do NOT re-count files here; take them from `validation.stack_counts` in the latest output-validator run (§1, or §3's re-run), which derived them deterministically from disk. The `library_count` vs per-library reference files and `integration_count` vs integration pair files checks surface as `field: "library_count"` / `field: "integration_count"` entries in `validation.stack_counts.issues[]` (absent when they agree); echo the exact numbers from `validation.stack_counts.observed` (`library_count_meta` / `ref_file_count`, `integration_count_meta` / `pair_file_count`). The `confidence_distribution`-sum equality is covered in §7.

Record each `validation.stack_counts.issues[]` count entry (`library_count` / `integration_count`) as a **WARNING** finding.

### 6. Validate Reference File Completeness

Record each `validation.stack_structure.issues[]` entry whose `check` is `references` as a **WARNING** finding.

### 7. Validate Confidence Tier Labels

Record each `validation.stack_structure.issues[]` entry whose `check` is `tier_labels` as a **WARNING** finding.

- [ ] metadata.json: `confidence_distribution` sums to `library_count`: take this from the latest output-validator run (`--skill-type stack`): the `field: "confidence_distribution"` entry in `validation.stack_counts.issues[]` is present only on mismatch, with `validation.stack_counts.observed.confidence_sum` vs `library_count_meta` for the exact numbers. Do not re-sum the distribution by hand.

Record the `validation.stack_counts` `confidence_distribution` issue, if any, as a **WARNING** finding.

### 8. Validate context-snippet.md

Take the first-line format, `|IMPORTANT:` second-line, and token-estimate checks from `validation.context_snippet.issues` returned by the §1 output-validator run — it performs the line-1 `[name vVersion]|root:` pattern match, the line-2 check, and the `len(content)//4` token estimate deterministically, so this step does not recompute them. Record each reported issue as a **WARNING** finding.

Then verify the two stack-specific rows the generic validator does not cover:
- [ ] Stack and integrations lines present
- [ ] Token estimate lands near the ~80-120 design target from step 7 §5 (the validator flags only its wider <40 / >300 bounds; an ~80-150 snippet with an overflow-strategy `workflow_warning` is expected, not a defect)

Record format violations as **WARNING** findings.

### 9. Security Scan

Run the scan whether or not skill-check is available. Resolve `{securityScanHelper}` ← first existing path in `{securityScanProbeOrder}` and run it on the committed package, with the longest timeout your shell tool takes (it ends within two minutes):

```bash
uv run {securityScanHelper} run --package "<skill-dir>"
```

It runs Snyk Agent Scan, at the version it pins, only when `SNYK_TOKEN` is set, and reads the scanner's own JSON report, so rely on the JSON it prints, never on an exit code. Record each `findings[]` entry's `text` as an advisory **WARNING** finding: security findings do not block the report. Record its `note` in the §10 validation results, as "security scan: {note}" for `pass` and `warn` and as "security scan skipped: {note}" for `skipped`. When no `{securityScanProbeOrder}` path exists, record no finding, only the §10 note "security scan skipped: skf-security-scan.py missing"; when the call prints no JSON object, the note "security scan skipped: skf-security-scan.py printed no result".

Bind `{security_result}` ← the helper's `security_result` (`pass`, `warn` or `skipped`), or `skipped` when the helper is missing or printed no JSON object. Step 9 puts it in the result envelope and the result contract.

### 10. Display Validation Results

Report the validation outcome. If all checks passed, state so and name what was verified: file presence (`{count}/{count}`), SKILL.md structure, metadata.json fields, the `{lib_count}` library + `{pair_count}` integration reference files, and complete confidence-tier coverage. If there were findings, report the `{warning_count}` finding(s), each with severity, description, and file path, plus files present/expected and warning/error counts; when errors include missing files, note this may indicate a write failure in step 07. In both cases, list each §9 security scan note.

### 11. Auto-Proceed to Next Step

Load, read the full file and then execute `{nextStepFile}`.

reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall