Skip to content

Adonis0123/adonis-skills / skills/agent-skill-evaluation

agent-skill-evaluation

Flagged

Use when comparing or trial-testing agent skills before installing — including GitHub/skills.sh URLs, 测试一下/值不值得装, package screening, and A/B artifact runs.

references/peer-review-cli.md

View on GitHub
# Peer-review CLI (skill install decisions)

Use after inventory + temp exercise, **before** presenting final A/B/C for non-trivial installs. Scope: **daily coding fit** across every agent host you use, not one host only. Owner account specifics: `references/local-owner-stack.md` (if present).

## Prompt package (always write under `/tmp/...-eval/`)

| File                                                | Content                                          |
| --------------------------------------------------- | ------------------------------------------------ |
| `PEER_REVIEW_PROMPT.md`                             | Locked evidence + A/B/C + required answer format |
| `EVAL.md`                                           | Your trial notes (optional attach)               |
| `peer-claude.md` / `peer-codex.md` / `peer-grok.md` | Raw peer outputs                                 |
| `PEER_SUMMARY.md`                                   | Choice table + consensus                         |

### Required answer format for peers

1. **Choice**: A / B / C
2. **One-line reason** (daily coding, not popularity)
3. **When it would become A** (concrete trigger or never)
4. **What to use instead** (≤3 bullets)
5. **Risk if installed** (≤2 bullets)

Forbid install counts / marketing as primary evidence.

## Claude (prefer a separate account / strongest model for second opinions)

```bash
# optional: point at a second account's config dir
export CLAUDE_CONFIG_DIR="${CLAUDE_CONFIG_DIR:-$HOME/.claude}"

claude -p --model <strongest-model> --max-turns 8 --output-format text \
  < /tmp/<eval>/PEER_REVIEW_PROMPT.md \
  > /tmp/<eval>/peer-claude.md
```

### Pitfalls

| Mistake                                                            | Effect                                                                                         | Fix                                              |
| ------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------- | ------------------------------------------------ |
| Pass prompt as shell argv with markdown `**` + `--allowedTools ''` | Shell word-splits; Claude treats `**foo**` as allow-rules; may error “Input must be provided…” | stdin or file only; omit empty `--allowedTools`  |
| Source the full shell rc from an agent gateway session             | May false-block on gateway restart heuristics                                                  | Set `CLAUDE_CONFIG_DIR` (+ needed env vars) only |
| Single-host framing in prompt                                      | Peers ignore Claude/Codex native subagents                                                     | Explicitly ask daily coding multi-tool fit       |

## Codex

```bash
codex exec --skip-git-repo-check -C /tmp/<eval> \
  "$(cat /tmp/<eval>/PEER_REVIEW_PROMPT.md)" \
  > /tmp/<eval>/peer-codex.md
```

- Prefer **default model** for ChatGPT-account Codex profiles; do not pin models that 400 (e.g. unsupported `o4-mini` on some accounts).
- Reply is often after echoed prompt — take the **last** Choice block.
- Auth/`token_revoked` / refresh invalidated → mark Codex **n/a**, keep remaining peers, still ship A/B/C. Do not retry login mid-eval or invent a Codex vote.

## Grok

```bash
# Prefer a signed-in home; a secondary home may be unsigned.
export GROK_HOME="${GROK_HOME:-$HOME/.grok}"

# Path containing the string "grok" can false-trigger gateway restart guards in some hosts.
# Safe pattern: copy binary to neutral path once per trial.
cp "$(command -v grok || echo "$HOME/.grok/bin/grok")" /tmp/<eval>/xai-cli
chmod +x /tmp/<eval>/xai-cli

/tmp/<eval>/xai-cli \
  --prompt-file /tmp/<eval>/PEER_REVIEW_PROMPT.md \
  --output-format json \
  --max-turns 4 \
  --cwd /tmp/<eval> \
  --always-approve \
  > /tmp/<eval>/peer-grok.md
```

### Pitfalls

| Mistake                                                              | Effect                           | Fix                                                     |
| -------------------------------------------------------------------- | -------------------------------- | ------------------------------------------------------- |
| Combine `--prompt-file` with `-p PROMPT`                             | CLI error: mutually exclusive    | One input path only                                     |
| Secondary `GROK_HOME` unsigned                                       | “Not signed in”                  | Fall back to signed profile; report which home was used |
| Invoke path literally named `*grok*` under some agent gateway guards | Command blocked as false restart | Neutral binary path under `/tmp/.../xai-cli`            |
| Parse only raw JSON file as user reply                               | Noisy                            | Extract `text` field for synthesis                      |

## Synthesis (you, after peers)

| Step       | Do                                                         |
| ---------- | ---------------------------------------------------------- |
| Table      | Peer → Choice → one-line                                   |
| Consensus  | Install or not; package vs single skill                    |
| Your A/B/C | Final call for the user; peers inform, you decide          |
| Status     | temporary-only / not installed / already present — precise |

Never install because peers disagreed mildly; default stays **B** unless a peer shows a **real coding-agent** gap the host cannot cover.
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall