majiayu000/spellbook / skills/codebase-audit
codebase-audit
Verified全面代码库审计 — 自适应并行深度分析(前后端契约、数据完整性、异常处理/安全、架构/技术债、配置/缓存),结构化 findings + 对抗验证 + 基线对比,输出按严重程度排序的统一报告和修复路线图。支持 quick 快速体检模式。Use when user asks to audit, analyze, or review an entire codebase for design issues, find hidden bugs, check architecture health, or asks '全面审查', '代码库审计', '分析设计问题', 'audit codebase', 'health check', '有哪些问题', '快速体检'. Also trigger when user asks to find silent degradation, data flow breakpoints, type mismatches between frontend and backend, or wants to understand technical debt across a project.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 11.1 KB |
| evals/README.md | markdown | 2.6 KB |
| evals/evals.json | config | 2.6 KB |
| evals/expected-findings.json | config | 3.6 KB |
| evals/fixture/README.md | markdown | 406 B |
| evals/fixture/app/__init__.py | script | 0 B |
| evals/fixture/app/config.py | script | 548 B |
| evals/fixture/app/handlers.py | script | 587 B |
| evals/fixture/app/main.py | script | 787 B |
| evals/fixture/app/pipeline.py | script | 807 B |
| evals/fixture/app/registry.py | script | 983 B |
| evals/fixture/app/storage.py | script | 669 B |
| evals/fixture/app/tasks.py | script | 510 B |
| evals/fixture/config.yaml | config | 72 B |
| evals/fixture/requirements.txt | manifest | 42 B |
| evals/fixture/tests/test_pipeline.py | script | 418 B |
| references/agent-prompts.md | markdown | 18.1 KB |
| references/ledger-format.md | markdown | 2.3 KB |
| references/stack-patterns.md | markdown | 2.8 KB |
| references/workflow-template.md | markdown | 5.2 KB |
evals/README.md
View on GitHub# Evals — Planted-Bug Fixture
`fixture/` is a deliberately buggy mini-project (Python backend, "sectionsvc"). Every bug is
intentional and catalogued in `expected-findings.json`. **Never "fix" fixture code** — it is eval
ground truth.
## Running an eval
1. Copy `fixture/` to an isolated directory outside this checkout. Run the skill against that
copy (see `evals.json` prompts). Point agents at the copy only — they must not read
`expected-findings.json` or this README (keep ground truth out of `{TARGET_DIR}`).
2. Score: match reported findings against `expected-findings.json` by **file + category**
(line numbers may drift). Pass = ≥ 8 of the 10 `must_detect` findings.
3. Findings 9 (test-quality) and 10 (concurrency) require the optional dimensions; if the run
didn't enable them, substitute bonus findings 11/12 into the must set per the notes.
4. Record the exact source commit, runtime/model, prompt, enabled dimensions, start/end time,
actual token/cost fields when available, and before/after target hashes. Score only after
the agent finishes. List matched findings, misses, unsupported claims and valid findings
outside the catalogue separately. Catalogue absence alone does not make a finding false.
5. For a comparison, use the same target, prompt, model and verification conditions in both
runs; load the skill only in the skill run. A single fixture run measures this task, not
external adoption or general model improvement. Missing usage/cost is unknown, not zero.
To run the fixture's own tests in isolation:
```bash
cd /absolute/path/to/the/isolated-fixture-copy
PYTHONDONTWRITEBYTECODE=1 uv run --with-requirements requirements.txt python -m pytest -p no:cacheprovider tests
```
## Reports and repeat runs
Evals 1–3 explicitly override the skill's default report and ledger writes with a no-write
request. Save their returned report outside the target and confirm target hashes did not
change. A normal skill invocation permits reports under the target and `.audit/` ledger
updates. Never edit or delete files in the original fixture as cleanup.
Eval 4 requires an explicit request to persist a report and ledger in the isolated copy on
both runs. Give reports distinct paths (`audit-report-run-1.md` and `audit-report-run-2.md`)
even when the runs share a date. Preserve that copy and both reports for checking stable IDs, `first_seen`, and
`still-open` classifications. Do not interpret a missed finding as resolved without reading
the affected source. The historical agent-count/output wording is not an acceptance gate;
local passes and explicitly authorized agents follow the current skill contract.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |