Adonis0123/adonis-skills / skills/agent-skill-evaluation
agent-skill-evaluation
FlaggedUse when comparing or trial-testing agent skills before installing — including GitHub/skills.sh URLs, 测试一下/值不值得装, package screening, and A/B artifact runs.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 30.3 KB |
| references/conceptual-multi-agent-skill-trial.md | markdown | 4.0 KB |
| references/image-producing-skill-trial.md | markdown | 2.7 KB |
| references/mattpocock-catalog-renames.md | markdown | 2.7 KB |
| references/mattpocock-shortlist-20260807.md | markdown | 3.2 KB |
| references/mattpocock-skills-screening.md | markdown | 15.3 KB |
| references/mattpocock-tdd-trial-20260807.md | markdown | 2.8 KB |
| references/mattpocock-tdd-trial-20260908.md | markdown | 2.5 KB |
| references/mattpocock-to-prd-trial-20260807.md | markdown | 1.8 KB |
| references/mattpocock-wayfinder-trial-20260909.md | markdown | 2.3 KB |
| references/output-style-skill-screening.md | markdown | 6.0 KB |
| references/peer-review-cli.md | markdown | 5.4 KB |
| references/single-skill-temp-trial.md | markdown | 7.7 KB |
| references/skill-eval-harness-trial.md | markdown | 3.6 KB |
| references/tdd-process-skill-trial.md | markdown | 2.9 KB |
| references/ui-design-ab-testing.md | markdown | 5.8 KB |
references/conceptual-multi-agent-skill-trial.md
View on GitHub# Conceptual multi-agent skill trials
Use when option **C** targets a skill whose value is _orchestration patterns / multi-agent taxonomy_, not a host-native tool path.
## Recognition signals
- SKILL.md centers sequential / parallel / hierarchical / consensus diagrams
- Ships `examples/*.py` + `scripts/*` but agents return fixed strings (no model/tool calls)
- “Framework support” is dict/dataclass templates (CrewAI, AutoGen, LangGraph, Swarm) without runnable deps
- Marketing: sophisticated multi-agent systems, production-ready, framework-agnostic
## Exercise checklist (temp clone only)
1. `git clone --depth 1` → `/tmp/<name>-eval` — never write enabled skill dirs.
2. Import and run pattern classes with correct constructors (read signatures; hierarchical often needs `Dict[team, List[agent]]`, consensus needs a mediator).
3. Run utilities: message broker, workflow deps / critical path, simple benchmarks.
4. Check demos: `python examples/*.py` exit 0 **and** whether `__main__` prints anything. Silent success ≠ exercised.
5. Classify each surface:
| Surface | Stub | Real |
| --------------------- | --------------- | ------------------------------------ |
| Pattern orchestrators | string `work()` | LLM/tool agents |
| Framework files | template dicts | importable framework runs |
| Host multi-agent | N/A | host delegate tool, spawn CLI agents |
6. Dual-track map against host stack before A/B/C:
| Host capability (example) | Job |
| ------------------------------------------ | --------------------------------- |
| host delegate tool (e.g. `delegate_task`) | real subagents, batch, background |
| `claude-code` / `codex` / `opencode` | external agent delegation |
| loop skills (`task-completion-loop`, etc.) | bounded multi-step orchestration |
## Decision bar
- **A full package** (e.g. `qodex-ai/ai-agent-skills` ~80 skills): almost never — selection tax + dual-track.
- **B keep current**: default when host already runs real multi-agent work.
- **C permanent single skill**: only if a real task needs the _patterns as code_ and mental load ≤ host path. Teaching/checklist alone → **no install**.
Install counts (skills.sh `userInteractionCount`) are discovery only.
## Baseline: qodex-ai multi-agent-orchestration (2026-08-04)
| Fact | Detail |
| -------------------------------- | ------------------------------------------------------------------------------ |
| URL | https://www.skills.sh/qodex-ai/ai-agent-skills/multi-agent-orchestration |
| Installs (ld+json) | 1957 at trial |
| Temp path | `/tmp/multi-agent-orch-eval` |
| Patterns exercised | sequential, parallel, hierarchical, consensus, DAG, adaptive — all stub agents |
| Utilities | broker / workflow / effectiveness metrics OK in-process |
| Examples CLI | exit 0, **no `__main__` demos** |
| Frameworks | templates only |
| Host multi-agent integration | none (no Claude/Codex/Grok subagents, no host delegate tool) |
| Peer review (daily coding scope) | Claude **C**; Codex **B**; Grok **B** — consensus: do not permanent-install |
| Decision | **B**; permanent install not recommended; package-wide A no |
Re-trial only if host lacks multi-agent execution _and_ a task needs reusable pattern code beyond native subagents / host delegate tools.
## Cleanup
```bash
rm -rf /tmp/multi-agent-orch-eval
# do not uninstall unrelated adopted skills
```
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |