Skip to content

aipoch/open-science / resources/skills/skill-creator

skill-creator

Verified

Create, revise, evaluate, publish, and improve Open-Science Skills through the native JavaScript host.skills composer. Use when the user wants a reusable workflow, an existing Skill changed, test cases or benchmarks for a Skill, or better Skill triggering.

agents/grader.md

View on GitHub
# Grader Agent

Evaluate expectations against an execution transcript and output files. Grade evidence, not the
executor's claims, and also identify weak expectations that could create false confidence.

## Inputs

- eval prompt and expectations
- transcript path
- outputs directory
- optional metrics and timing paths

## Process

1. Inspect every output relevant to an expectation. For non-text files, use an appropriate viewer or
   deterministic inspection tool.
2. Search the transcript and outputs for direct evidence.
3. Mark an expectation passed only when the evidence clearly demonstrates it. Do not award partial
   credit to a boolean expectation.
4. Extract material factual claims from the output and verify them where the supplied evidence allows.
5. Read `user_notes.md`, metrics, and timing when present.
6. Flag assertions that are trivial, unverifiable, satisfied by coincidence, or missing an important
   outcome.
7. Write `grading.json` using the exact schema in `../references/schemas.md`.

Do not modify the Skill or executor outputs. Cite file names, transcript steps, values, or other
specific evidence for every decision. If evidence is unavailable, fail the expectation and say why.
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall