mgiovani/ai-arsenal / skills/docs-check
docs-check
VerifiedRead-only audit of documentation against the current codebase, flags stale docs, missing sections, broken links, and hallucinated claims (wrong file references, wrong counts, diagram entities that don't exist in code). Use for "check the docs", "audit documentation", "are the docs stale", "find hallucinations in docs", "docs health check", "does this doc still match the code", or before onboarding/release. Reports only, never edits files, for actually fixing or regenerating docs use docs-update instead.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 8.0 KB |
| evals/evals.json | config | 3.6 KB |
| evals/trigger-eval.json | config | 1.9 KB |
| references/scoring-criteria.md | markdown | 2.0 KB |
| references/verification-patterns.md | markdown | 4.8 KB |
references/scoring-criteria.md
View on GitHub# Rating Criteria
Coarse per-document rubric for Phase 5. No numeric scoring or weighted averages: one rating per document, chosen by the first rule below that applies (checked in order).
## Rating Definitions
| Rating | Applies when |
|---------|--------------|
| Broken | Contains a hallucination (claim contradicted by the codebase), a broken internal link, or invalid Mermaid syntax |
| Missing | Doc is expected given the detected stack (e.g. a `data-model.md` for a repo with a schema) but doesn't exist |
| Stale | All claims check out, but the doc predates a significant related code change, has unreplaced `{{PLACEHOLDER}}` values, or is missing an expected section |
| Good | Current, complete, no broken links, no invalid diagrams, no hallucinations |
Broken outranks Stale: a doc with one false claim is Broken even if everything else about it is current. A wrong claim is worse than an old-but-true one.
## Evidence Requirement
Every rating must cite what was actually checked: a git log date, a grep result, a find count. Never assign "Stale" or "Broken" without the command output that justifies it; never assign "Good" without having actually run the freshness/completeness/quality checks in Phase 4.
## Detailed Report Format (per document)
For each documentation file rated Stale, Broken, or Missing, include:
- Filename and path
- Rating and the one-line reason
- Specific issues with line numbers where applicable
- The verification command used as evidence
- Recommended follow-up command (docs-update, docs-diagram, docs-init)
Good-rated docs need only a one-line listing: no evidence dump for docs with nothing wrong.
## Best Practices
- Run regularly: freshness drift compounds silently between checks.
- Prioritize Broken over Stale over Missing when recommending fixes: hallucinations mislead readers actively, staleness merely under-informs them.
- Don't let a "mostly fine" rating hide one hallucinated claim inside a doc: Broken always surfaces in the summary even if the rest of the doc is solid.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |