Skip to content

moonlight-lupin/agent-skills / software-development/skill-quality-review

skill-quality-review

Verified

Use when auditing or reviewing skill quality — batch checks across a library, deep per-skill grep methodology, usage stats, efficacy testing, or fix planning. Single consolidated home for the writing-for-agents review framework and the 3-surface review methodology.

templates/efficacy-rubric-template.md

View on GitHub
# Efficacy Rubric Template

Copy per test. Fill one block per scenario. Scenarios must be verbatim-identical across arms;
target each rubric at a change the refactor made (pointer firing, dedupe, noise rejection, new-rule recall).

```markdown
# <Skill> A/B — Scenarios & Rubric

Controlled test: old vs refactored <skill>. Same model, same scenario prompts. Only variable: skill version.
Each child reads ONE arm's files and answers its scenarios in order, using only the material.

### S<n> — <what the refactor changed, e.g. pointer efficacy>
<verbatim scenario prompt — identical text in both dispatches>

Rubric (1 pt each):
- a) <checkable behavior point>
- b) <arm-discriminating point, e.g. consults references/<file>.md — measures pointer firing>
- c) <noise-rejection point — does NOT do the counterproductive thing>

## Scoring protocol
- One child per arm, identical goal text, path is the only context difference.
- Score each cell 0-3 (a/b/c). Report per-scenario and aggregate.
- State limits: n=1 per cell is directional; measures single-turn rule recall, not end-to-end outcomes.
```
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 196kUpdated Claude CodeCodexVerifiedInstall