hashgraph-online/awesome-codex-plugins / plugins/boshu2/agentops/skills/skill-eval
skill-eval
VerifiedMeasure whether a skill helps by comparing runs with and without it. Use when: reading skill A/B results or deciding to keep, revise or remove one.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 15.5 KB |
| references/behavioral-probes.md | markdown | 4.7 KB |
| references/coding-memory-readout.md | markdown | 2.9 KB |
| references/dialogue-skills.md | markdown | 1.2 KB |
| references/seeding.md | markdown | 5.7 KB |
references/dialogue-skills.md
View on GitHub# Dialogue skills
Some skills reach their result through the caller's answers: an interview, a
brainstorm or a clarifying loop. A single-turn case shows only the opening
move, such as whether the first reply asks one question. It cannot show needs
the dialogue missed or questions it should never have asked.
- **Keep it away from real people.** Do not run the skill in a session where
its questions reach the caller or anyone else. Their answers are not a fixed
case, and the run interrupts their work.
- **Answer with a stand-in user.** Use an authorized runner that answers each
turn as a stand-in with a goal, needs it reveals only when the right question
reaches it, and a point past which more questions are overkill. Give both
arms the same stand-ins and cases.
- **Grade blind against fixed criteria.** Fix the criteria before the run and
grade transcripts without knowing the arm, so one grade shows both missed
needs and needless questions. Calibrate this grader like any other.
- **Know the stand-in's bias.** A stand-in that defers every choice up front
shows the skill's recommendations but overstates how far scope grows.
With no such runner, list the dialogue under **Not proven**; do not build one
to rescue the measurement.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |