Skip to content

hashgraph-online/awesome-codex-plugins / plugins/boshu2/agentops/skills/skill-eval

skill-eval

Verified

Measure whether a skill helps by comparing runs with and without it. Use when: reading skill A/B results or deciding to keep, revise or remove one.

FileKindSize
SKILL.mdskill md15.5 KB
references/behavioral-probes.mdmarkdown4.7 KB
references/coding-memory-readout.mdmarkdown2.9 KB
references/dialogue-skills.mdmarkdown1.2 KB
references/seeding.mdmarkdown5.7 KB

references/dialogue-skills.md

View on GitHub
# Dialogue skills

Some skills reach their result through the caller's answers: an interview, a
brainstorm or a clarifying loop. A single-turn case shows only the opening
move, such as whether the first reply asks one question. It cannot show needs
the dialogue missed or questions it should never have asked.

- **Keep it away from real people.** Do not run the skill in a session where
  its questions reach the caller or anyone else. Their answers are not a fixed
  case, and the run interrupts their work.
- **Answer with a stand-in user.** Use an authorized runner that answers each
  turn as a stand-in with a goal, needs it reveals only when the right question
  reaches it, and a point past which more questions are overkill. Give both
  arms the same stand-ins and cases.
- **Grade blind against fixed criteria.** Fix the criteria before the run and
  grade transcripts without knowing the arm, so one grade shows both missed
  needs and needless questions. Calibrate this grader like any other.
- **Know the stand-in's bias.** A stand-in that defers every choice up front
  shows the skill's recommendations but overstates how far scope grows.

With no such runner, list the dialogue under **Not proven**; do not build one
to rescue the measurement.
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall