Adonis0123/adonis-skills / skills/agent-skill-evaluation
agent-skill-evaluation
FlaggedUse when comparing or trial-testing agent skills before installing — including GitHub/skills.sh URLs, 测试一下/值不值得装, package screening, and A/B artifact runs.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 30.3 KB |
| references/conceptual-multi-agent-skill-trial.md | markdown | 4.0 KB |
| references/image-producing-skill-trial.md | markdown | 2.7 KB |
| references/mattpocock-catalog-renames.md | markdown | 2.7 KB |
| references/mattpocock-shortlist-20260807.md | markdown | 3.2 KB |
| references/mattpocock-skills-screening.md | markdown | 15.3 KB |
| references/mattpocock-tdd-trial-20260807.md | markdown | 2.8 KB |
| references/mattpocock-tdd-trial-20260908.md | markdown | 2.5 KB |
| references/mattpocock-to-prd-trial-20260807.md | markdown | 1.8 KB |
| references/mattpocock-wayfinder-trial-20260909.md | markdown | 2.3 KB |
| references/output-style-skill-screening.md | markdown | 6.0 KB |
| references/peer-review-cli.md | markdown | 5.4 KB |
| references/single-skill-temp-trial.md | markdown | 7.7 KB |
| references/skill-eval-harness-trial.md | markdown | 3.6 KB |
| references/tdd-process-skill-trial.md | markdown | 2.9 KB |
| references/ui-design-ab-testing.md | markdown | 5.8 KB |
references/single-skill-temp-trial.md
View on GitHub# Single-skill temp trial(URL / 测试一下)
For one skill link or `owner/repo@skill` when the user wants **evaluation**, not install.
## Triggers (this route)
- GitHub tree/blob under `…/skills/<name>`
- skills.sh page for one skill
- 测试一下 / 试一下 / 评估 / 值不值得装 + skill name or URL
- Explicit “先别装,先试”
Not this route: pure catalog discovery with no install judgment (may still use find-skills **search only**).
## Layout
```text
/tmp/<skill-or-pkg>-eval-YYYYMMDD/
upstream/ # optional full clone --depth 1
<skill-name>/ # copy of skill dir only
trial-artifact/ # exercise outputs (design, HTML, cited md)
EVAL.md # inventory + dual-track + exercise notes
PEER_REVIEW_PROMPT.md # if peers run
peer-*.md / PEER_SUMMARY.md
```
## Steps (minimum)
1. Confirm target skill name + upstream repo.
2. Clone or sparse-copy **only** that skill into `/tmp/...` (no write into global skills directories).
3. Light inventory:
- same name local? (every global skills directory, e.g. `~/.agents/skills`)
- hash of `SKILL.md` vs upstream tip
- local alternatives (different name, same job)
4. Dual-track map (process skills especially): discuss/plan/grill/lite-code/debug stacks.
5. Exercise:
- **Process / design-gate:** one locked brief → sample design or decision artifact under `trial-artifact/`; if scripts/companion exist, start→HTTP smoke→stop under temp `project-dir`.
- **Research-thin:** one cited markdown under temp.
- **Research pack (1 router + N specialists):** follow **Research-pack smoke** below — do not treat the repo as one skill and do not run the full volume gate.
- **UI:** follow main A/B artifact path when comparing two.
6. Non-trivial → peers (`peer-review-cli.md`); Codex usage-limit → n/a, still ship A/B/C.
7. Deliver: Choice A/B/C for **this skill**, one-line reason, when→upgrade, use-instead ≤3, risks ≤2, install status precise.
## Research-pack smoke (1 router + N specialists)
When the GitHub URL is a **repo root** (or a `skills/` tree) whose `SKILL.md` names are a router plus TAM / chain / competition / finance / valuation specialists:
1. **Classify as package.** Inventory every `skills/*/SKILL.md`. Dual-track against local news/citation/digest skills. Package **A** (enable all) is almost never the 测试 outcome.
2. **Lock one theme the user already named** (e.g. a product category). That theme is the opening record. For the trial only, treat it as 「无需提问,直接开始」 — do not stop the eval on the router’s interview HARD-GATE. A real research engagement still owes that interview.
3. **C-smoke 1–2 specialists that match the theme**, using the pack’s own **单独专项 / 简版 / 有限披露** path. Typical pair: market-sizing + competitive-landscape. Copy those skill dirs under the eval tree; write artifacts under `trial-artifact/` in the pack’s project-folder shape if it has one.
4. **Do not** execute the pack’s full 8–13 module volume (multi-thousand-char specialists + a 25k–60k synthesis) as a 测试. Write the downgrade on the 交付清单 so the trial is not labelled 完整行业研究.
5. **Exercise the method that can fail**, not the template headings: keep conflicting TAM vs company ARR as a range; never average unlike denominators; never compute CR/HHI without a compatible market total; ARR ≠ FY revenue; user counts ≠ share. If the specialist defaults to a formula `.xlsx`, write real formulas (temp Python deps **inside** the eval tree) or mark the workbook 待生成 — a static markdown table is not a completed sizing trial.
6. **Score intervention cost** as: how much of the pack you had to override (opening Q&A, volume gate, Excel toolchain) to finish a chat-sized 测试. High override → lean **B**, keep **C** for a later real TAM/格局 report.
7. Deliver package A/B/C plus which **one** specialist would be the C copy if they later ask to install something.
## Accidental global install recovery(硬)
If `npx skills add -g` / enable already happened mid-eval:
1. `npx skills remove <skill> -g -y` (or your skill manager's remove, after showing the plan).
2. Verify gone: `~/.agents/skills/<name>` (or your global dir), per-host symlinks, `.skill-lock.json` entry.
3. Say so in the user-facing status (“误装已卸”).
4. Continue only from `/tmp` trial; do not treat the brief enable as adoption.
## Decision defaults (example)
| Finding | Lean |
| ------------------------------------------------------------------- | ----------------------------------------------------------------------- |
| Local alternative covers job; candidate adds ceremony | **B** |
| Unique tool (e.g. visual companion) but daily path is lite surgical | **B**, C only if that gap recurs |
| Hash-match already installed | **B** (zero delta) |
| Real gap + lower mental load after one real task | single-skill **A** after user says install |
| Whole package missing many skills user did not ask for | package **A** almost never |
| 1+N research pack; daily job is a quick chat 「查一下竞品」 | **B**; C = one specialist copy when a real TAM/格局 report is requested |
## Baseline: `obra/superpowers@brainstorming`(2026-08-07)
| Item | Note |
| ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Job | Design-before-code; one Q at a time; 2–3 approaches; spec path; handoff only to `writing-plans` |
| Unique | Browser visual companion (`scripts/start-server.sh`, key-gated URL) |
| Already from same pkg (hash match) | `systematic-debugging`, `test-driven-development`, `writing-plans` |
| Local overlap | `discuss-before-plan`, `grilling`/`grill-me`/`grill-with-docs`, `workflow-gate` creative→Brainstorm pointer, a lightweight surgical-change path |
| Friction | HARD-GATE on every creative/behavior change; default `docs/superpowers/specs/` + commit; dual-track with discuss-before-plan |
| Trial verdict | **B** (not worth permanent install for daily surgical product work). Package full add rejected. Optional later **C**/single A only if visual companion gap is proven. |
| Side finding | `workflow-gate` may point at missing `brainstorming` — fix gate target to `discuss-before-plan` is a separate edit; not a reason to force-install brainstorming. |
Re-inventory before revisiting; do not treat this baseline as forever.
## Deliverable shape (chat-friendly)
```text
结论:B|C|A(单 skill)— …
纠正/状态:temporary-only | not installed | 误装已卸
证据:/tmp/… 路径 + 是否实跑
替代:≤3
风险:≤2
```
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |