Adonis0123/adonis-skills / skills/agent-skill-evaluation
agent-skill-evaluation
FlaggedUse when comparing or trial-testing agent skills before installing — including GitHub/skills.sh URLs, 测试一下/值不值得装, package screening, and A/B artifact runs.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 30.3 KB |
| references/conceptual-multi-agent-skill-trial.md | markdown | 4.0 KB |
| references/image-producing-skill-trial.md | markdown | 2.7 KB |
| references/mattpocock-catalog-renames.md | markdown | 2.7 KB |
| references/mattpocock-shortlist-20260807.md | markdown | 3.2 KB |
| references/mattpocock-skills-screening.md | markdown | 15.3 KB |
| references/mattpocock-tdd-trial-20260807.md | markdown | 2.8 KB |
| references/mattpocock-tdd-trial-20260908.md | markdown | 2.5 KB |
| references/mattpocock-to-prd-trial-20260807.md | markdown | 1.8 KB |
| references/mattpocock-wayfinder-trial-20260909.md | markdown | 2.3 KB |
| references/output-style-skill-screening.md | markdown | 6.0 KB |
| references/peer-review-cli.md | markdown | 5.4 KB |
| references/single-skill-temp-trial.md | markdown | 7.7 KB |
| references/skill-eval-harness-trial.md | markdown | 3.6 KB |
| references/tdd-process-skill-trial.md | markdown | 2.9 KB |
| references/ui-design-ab-testing.md | markdown | 5.8 KB |
references/mattpocock-skills-screening.md
View on GitHub# mattpocock/skills — package screening notes
Source of truth for **whether to full-install** this package on the owner’s machines.
Not a substitute for re-inventory: re-clone and re-hash if evaluating months later.
## What the package is
- Repo: https://github.com/mattpocock/skills
- Philosophy: small, composable skills; explicitly against owning the whole process like GSD / BMAD / Spec-Kit.
- Main engineering chain (user-invoked):
`grill-me` / `grill-with-docs` → `to-spec` → `to-tickets` → `tdd` → `improve-codebase-architecture`
- Setup entry: `setup-matt-pocock-skills` (issue tracker + triage labels + domain doc layout under `docs/agents/*`, and an `## Agent skills` block in `CLAUDE.md`/`AGENTS.md`).
## Install surfaces (do not double-install)
| Path | Effect |
| ----------------------------------------- | ----------------------------------------------------- |
| Claude Code plugin `mattpocock-skills` | Managed bundle; auto-updates |
| `npx skills@latest add mattpocock/skills` | Copies editable skills into project/global agent dirs |
| Both | Upstream warns: **every skill appears twice** |
CLI useful flags (skills@latest):
```bash
npx skills@latest add mattpocock/skills -l # list only
npx skills@latest use mattpocock/skills@<skill> # prompt only, no install
npx skills@latest add mattpocock/skills -s <name> -g # single skill global (only after user decision)
```
## Example owner baseline (verified 2026-08-03)
Already installed in the global skills directory with **SKILL.md hash == upstream tip** at evaluation time:
| Skill | Role |
| ------------------------------- | ------------------------------------------- |
| `grill-me` | User entry → grilling |
| `grilling` | Relentless interview loop |
| `grill-with-docs` | Grill + domain docs / ADR |
| `to-spec` | Conversation → spec (expects setup tracker) |
| `improve-codebase-architecture` | Deepening opportunities |
| `domain-modeling` | Glossary / CONTEXT sharpening |
| `codebase-design` | Deep-module vocabulary |
| `handoff` | Session handoff doc |
| `teach` | Multi-session teaching |
Lock provenance also pointed at `source: mattpocock/skills` for the above.
### Not installed (and usually should stay off full-add)
| Skill | Why careful |
| -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `setup-matt-pocock-skills` | Rewrites agent docs; forces tracker/domain layout |
| `to-tickets` / `triage` | Dual-track vs the primary tracker + chat intake + code-host MR flow |
| `tdd` | Overlaps local `test-driven-development` (much heavier) |
| `diagnosing-bugs` | Real possible gap; may trial alone vs `systematic-debugging` |
| `code-review` | Overlaps existing CR stack; often wants issue-tracker config |
| `implement` / `prototype` / `ask-matt` | No proven gap; adds selection tax |
| `implement-spec` | Do not install. Orchestrator only: setup gate, then per-ticket worktree calling `tdd`, a merger, and `code-review`. A throwaway graph that commits is not evidence the shell runs without those missing skills. Second queue vs the primary tracker. Use `task-completion-loop`, a lightweight surgical-change path, or `writing-plans` + local `test-driven-development`. |
| `wayfinder` | **Trialed 2026-09-09 — B, do not install** (`references/mattpocock-wayfinder-trial-20260909.md`); tracker-coupled decision map vs the primary tracker + discuss-before-plan |
| `research` | **Trialed 2026-08-03 — do not install by default** (see below) |
| `loop-me` | **`in-progress/`**; trialed 2026-08-15 — do not install (see below) |
## Dual-track / coupling map (example)
| Upstream assumption | Local reality | Full install effect |
| --------------------------------------- | ---------------------------------------- | ----------------------------------------------------------- |
| GitHub/GitLab/local markdown issues | Another tracker is primary | Half-broken tracker skills or second queue |
| Root `CONTEXT.md` + `docs/adr/` default | Per-repo docs; avoid auto-create CONTEXT | Layout fight |
| `ready-for-agent` triage labels | Chat intake → tracker labels / owners | Parallel triage vocabulary |
| Composable skills still share setup | Multi-host skill budget already large | More trigger noise; Codex skill-description budget pressure |
## Recommended decision pattern
1. **Re-inventory** (clone + name diff + hash) — do not trust this note alone after long gaps.
2. If core list still hash-matches → **B: do not full install**.
3. If a **single** zero-setup skill has a proven gap (historically: `diagnosing-bugs`) → **C: temp trial only**.
4. Admission for permanent single install: real task evidence, ≤ mental load of local alternative, no setup side effects.
5. **Never A** solely because the package is popular or README is good.
## Minimal temp trial (diagnosing-bugs example)
```bash
# Prompt-only (preferred when CLI available)
npx skills@latest use mattpocock/skills@diagnosing-bugs
# Or file-only under /tmp
mkdir -p /tmp/mp-trial
cp -R /tmp/mattpocock-skills-eval/skills/engineering/diagnosing-bugs /tmp/mp-trial/
# feed that SKILL.md to the agent; do NOT run setup
```
Pass bar: must produce a **already-run red-capable repro command** before hypotheses; clean debug markers after fix; no new `docs/agents/*`.
## `research` single-skill trial (2026-08-03)
User re-scoped: evaluate **only** `research`, not the full package.
| Fact | Detail |
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Skill size | ~12-line body: background agent + primary sources + one cited Markdown + save where repo keeps notes |
| Coupling | **Zero** setup / tracker / CONTEXT force-create |
| Local overlap | Host already has web search / fetch + a standing “primary sources” rule |
| Trial method | `/tmp/mp-research-trial` with skill copied under `.claude/skills/research/`; produced `docs/research/2026-08-03-agent-skills-skillmd-frontmatter.md` from agentskills.io + Claude Platform primary docs |
| Install decision | **B for package, no permanent install of `research`** — thin checklist, not new capability |
| Cleanup | User asked to delete trial artifacts; removed `/tmp/mp-research-trial` (+ package eval temps). Did **not** touch installed Matt subset |
Re-trial only if a real task shows cited-note discipline failing without the skill. Prefer remembering the checklist over occupying a skill slot.
## `wait-what` + `writing-beats` trial (2026-08-22)
User asked: skills.sh URLs + 测试一下 / 值不值得装. First turn accidentally `npx skills add -g` — **removed** (`npx skills remove … -g -y` + leftover dirs). Re-ran as `/tmp` eval.
| Fact | Detail |
| ------------- | --------------------------------------------------------------------------------------------------------------- |
| Layout | `/tmp/mp-wait-beats-eval-20260822/` — hashes match upstream tip |
| wait-what | 3-line user-invoked re-pitch; STE100 + CONTEXT.md; slash-only |
| writing-beats | `skills/in-progress/`; CYOA one-beat-then-stop + grounding |
| Dual-track | wait-what → profile plain-language rules;writing-beats → report-writing / documentation-writer / writing-plans |
| Exercise | jargon dump vs re-pitch; raw pile → only beat 1 on disk |
| Peers | Claude (Opus) / Codex / Grok all **B+B** |
| Install | **B for both**. Package A still out of scope. Do not leave global copies. |
Re-trial wait-what only if the agent’s 「说人话」 still dumps jargon ≥3 times after a profile line. Re-trial writing-beats only after it leaves `in-progress/` **and** a real IDE long-form session wants beat-by-beat picks.
## `loop-me` single-skill trial (2026-08-15)
User asked: `npx skills add … --skill loop-me` + 测试/值不值得装. **Did not add.** Temp clone only.
| Fact | Detail |
| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| Bucket | `skills/in-progress/` — excluded from plugin + top README; can vanish |
| Size | 32-line wrapper around `/grilling` + vocab Trigger / Checkpoint / Push-right / Brief; workspace `workflows/*.md` + `NOTES.md` |
| Coupling | Zero setup/tracker/CONTEXT. High process coupling to already-installed `grilling` |
| Local overlap | `grilling`/`grill-me`/`grill-with-docs` + `discuss-before-plan` + `workflow-gate` + host cron / a daily digest skill / `goal-gate` |
| Trial | `/tmp/loop-me-eval-20260815/` — specified an existing 09:00 digest job as a loop; substance came from the digest skill, not the 32 lines |
| Peers | Claude (Opus) / Codex / Grok all **B** |
| Install decision | **B** — no permanent install. Package A still out of scope |
Re-trial only if the owner adopts `workflows/*.md` as cross-session SoT **and** 2–3 real unspecced life-loops fail under grilling + cron + the primary tracker.
## Cleanup checklist (after any mattpocock temp eval)
```bash
rm -rf /tmp/mp-research-trial /tmp/mattpocock-skills-eval /tmp/mattpocock-skills-peer-review /tmp/mp-trial /tmp/loop-me-eval-20260815
# leave installed grill-me, to-spec, ... alone unless user asks uninstall
```
## Peer-review note
When multi-agent review is requested (Claude / Codex / Grok), pass the inventory + coupling map from the main skill’s **Package Install Screening** section. Peers without disk access must be told what was hash-verified vs assumed.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |