Adonis0123/adonis-skills / skills/agent-skill-evaluation
agent-skill-evaluation
FlaggedUse when comparing or trial-testing agent skills before installing — including GitHub/skills.sh URLs, 测试一下/值不值得装, package screening, and A/B artifact runs.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 30.3 KB |
| references/conceptual-multi-agent-skill-trial.md | markdown | 4.0 KB |
| references/image-producing-skill-trial.md | markdown | 2.7 KB |
| references/mattpocock-catalog-renames.md | markdown | 2.7 KB |
| references/mattpocock-shortlist-20260807.md | markdown | 3.2 KB |
| references/mattpocock-skills-screening.md | markdown | 15.3 KB |
| references/mattpocock-tdd-trial-20260807.md | markdown | 2.8 KB |
| references/mattpocock-tdd-trial-20260908.md | markdown | 2.5 KB |
| references/mattpocock-to-prd-trial-20260807.md | markdown | 1.8 KB |
| references/mattpocock-wayfinder-trial-20260909.md | markdown | 2.3 KB |
| references/output-style-skill-screening.md | markdown | 6.0 KB |
| references/peer-review-cli.md | markdown | 5.4 KB |
| references/single-skill-temp-trial.md | markdown | 7.7 KB |
| references/skill-eval-harness-trial.md | markdown | 3.6 KB |
| references/tdd-process-skill-trial.md | markdown | 2.9 KB |
| references/ui-design-ab-testing.md | markdown | 5.8 KB |
references/peer-review-cli.md
View on GitHub# Peer-review CLI (skill install decisions)
Use after inventory + temp exercise, **before** presenting final A/B/C for non-trivial installs. Scope: **daily coding fit** across every agent host you use, not one host only. Owner account specifics: `references/local-owner-stack.md` (if present).
## Prompt package (always write under `/tmp/...-eval/`)
| File | Content |
| --------------------------------------------------- | ------------------------------------------------ |
| `PEER_REVIEW_PROMPT.md` | Locked evidence + A/B/C + required answer format |
| `EVAL.md` | Your trial notes (optional attach) |
| `peer-claude.md` / `peer-codex.md` / `peer-grok.md` | Raw peer outputs |
| `PEER_SUMMARY.md` | Choice table + consensus |
### Required answer format for peers
1. **Choice**: A / B / C
2. **One-line reason** (daily coding, not popularity)
3. **When it would become A** (concrete trigger or never)
4. **What to use instead** (≤3 bullets)
5. **Risk if installed** (≤2 bullets)
Forbid install counts / marketing as primary evidence.
## Claude (prefer a separate account / strongest model for second opinions)
```bash
# optional: point at a second account's config dir
export CLAUDE_CONFIG_DIR="${CLAUDE_CONFIG_DIR:-$HOME/.claude}"
claude -p --model <strongest-model> --max-turns 8 --output-format text \
< /tmp/<eval>/PEER_REVIEW_PROMPT.md \
> /tmp/<eval>/peer-claude.md
```
### Pitfalls
| Mistake | Effect | Fix |
| ------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------- | ------------------------------------------------ |
| Pass prompt as shell argv with markdown `**` + `--allowedTools ''` | Shell word-splits; Claude treats `**foo**` as allow-rules; may error “Input must be provided…” | stdin or file only; omit empty `--allowedTools` |
| Source the full shell rc from an agent gateway session | May false-block on gateway restart heuristics | Set `CLAUDE_CONFIG_DIR` (+ needed env vars) only |
| Single-host framing in prompt | Peers ignore Claude/Codex native subagents | Explicitly ask daily coding multi-tool fit |
## Codex
```bash
codex exec --skip-git-repo-check -C /tmp/<eval> \
"$(cat /tmp/<eval>/PEER_REVIEW_PROMPT.md)" \
> /tmp/<eval>/peer-codex.md
```
- Prefer **default model** for ChatGPT-account Codex profiles; do not pin models that 400 (e.g. unsupported `o4-mini` on some accounts).
- Reply is often after echoed prompt — take the **last** Choice block.
- Auth/`token_revoked` / refresh invalidated → mark Codex **n/a**, keep remaining peers, still ship A/B/C. Do not retry login mid-eval or invent a Codex vote.
## Grok
```bash
# Prefer a signed-in home; a secondary home may be unsigned.
export GROK_HOME="${GROK_HOME:-$HOME/.grok}"
# Path containing the string "grok" can false-trigger gateway restart guards in some hosts.
# Safe pattern: copy binary to neutral path once per trial.
cp "$(command -v grok || echo "$HOME/.grok/bin/grok")" /tmp/<eval>/xai-cli
chmod +x /tmp/<eval>/xai-cli
/tmp/<eval>/xai-cli \
--prompt-file /tmp/<eval>/PEER_REVIEW_PROMPT.md \
--output-format json \
--max-turns 4 \
--cwd /tmp/<eval> \
--always-approve \
> /tmp/<eval>/peer-grok.md
```
### Pitfalls
| Mistake | Effect | Fix |
| -------------------------------------------------------------------- | -------------------------------- | ------------------------------------------------------- |
| Combine `--prompt-file` with `-p PROMPT` | CLI error: mutually exclusive | One input path only |
| Secondary `GROK_HOME` unsigned | “Not signed in” | Fall back to signed profile; report which home was used |
| Invoke path literally named `*grok*` under some agent gateway guards | Command blocked as false restart | Neutral binary path under `/tmp/.../xai-cli` |
| Parse only raw JSON file as user reply | Noisy | Extract `text` field for synthesis |
## Synthesis (you, after peers)
| Step | Do |
| ---------- | ---------------------------------------------------------- |
| Table | Peer → Choice → one-line |
| Consensus | Install or not; package vs single skill |
| Your A/B/C | Final call for the user; peers inform, you decide |
| Status | temporary-only / not installed / already present — precise |
Never install because peers disagreed mildly; default stays **B** unless a peer shows a **real coding-agent** gap the host cannot cover.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |