aipoch/open-science / resources/skills/skill-creator
skill-creator
VerifiedCreate, revise, evaluate, publish, and improve Open-Science Skills through the native JavaScript host.skills composer. Use when the user wants a reusable workflow, an existing Skill changed, test cases or benchmarks for a Skill, or better Skill triggering.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 6.5 KB |
| agents/analyzer.md | markdown | 989 B |
| agents/comparator.md | markdown | 815 B |
| agents/grader.md | markdown | 1.2 KB |
| assets/eval_review.html | other | 3.5 KB |
| eval-viewer/generate-review.js | script | 4.0 KB |
| eval-viewer/viewer.html | other | 4.6 KB |
| package.json | manifest | 25 B |
| references/schemas.md | markdown | 3.7 KB |
| scripts/aggregate-benchmark.js | script | 1.2 KB |
| scripts/generate-report.js | script | 1.4 KB |
| scripts/improve-description.js | script | 1.6 KB |
| scripts/index.js | script | 321 B |
| scripts/quick-validate.js | script | 1.9 KB |
| scripts/run-eval.js | script | 395 B |
| scripts/run-loop.js | script | 793 B |
| scripts/utils.js | script | 1.0 KB |
agents/analyzer.md
View on GitHub# Post-hoc Analyzer Agent
After a valid blind comparison, unblind the result and explain why the winner performed better. Turn
the evidence into generalizable Skill improvements rather than copying one output.
## Process
1. Read `comparison.json` and the A/B mapping.
2. Read both Skill snapshots and their referenced resources.
3. Compare transcripts: instruction following, tool usage, errors, recovery, and unnecessary work.
4. Identify winner strengths and loser weaknesses that plausibly caused the observed difference.
5. Separate causal evidence from incidental differences.
6. Propose prioritized changes to instructions, scripts, examples, or edge-case handling.
7. Write `analysis.json` using the schema in `../references/schemas.md`.
For benchmark-only analysis, look for non-discriminating expectations, high-variance cases,
configuration failures, and quality/time/token trade-offs. Do not make subjective quality claims that
are unsupported by outputs or human feedback.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |