notque/vexjoy-agent / skills/meta/toolkit
toolkit
VerifiedToolkit management: create and evaluate skills and agents, manage routing tables, generate Claude.md.
references/agent-evaluation/common-issues.md
View on GitHub# Common Evaluation Issues
Separate deterministic failures from qualitative findings. Only the eight checks in `score-component.py` affect the structural score.
## Deterministic Failures
| Check | Common cause | Fix |
|---|---|---|
| Valid YAML frontmatter | Missing delimiters, invalid YAML, empty `name` or `description` | Repair the frontmatter and parse it again |
| Referenced files exist | Backtick-quoted path is stale or resolves from the wrong base | Correct the path or remove the stale claim |
| Patterns section | No heading containing `pattern` | Add a useful patterns section only when the component needs one |
| Error handling section | No heading containing `error` or `failure mode` | Document concrete recovery paths |
| Registered in routing | Missing from the applicable index or `/do` routing | Register the component through the normal index workflow |
| Reference files | No `references/` directory | Add references when progressive disclosure is useful; do not add empty padding |
| Workflow instructions | Missing `Instructions`, numbered Phase/Step heading, or `**Gate**` | Add the missing execution structure |
| No broken internal links | Relative Markdown links do not resolve | Repair or remove each broken link |
## Qualitative Findings
These do not change the structural score:
- **Stale guidance**: Instructions disagree with current code or repository policy.
- **Thin domain knowledge**: The component repeats generic advice without task-specific constraints or failure modes.
- **Unnecessary bulk**: Long examples, repeated rules, or catalogs crowd the entrypoint without changing behavior.
- **Tool mismatch**: Instructions require capabilities the declared tool set cannot provide.
- **Weak recovery guidance**: An error heading exists, but the listed action does not let the operator recover.
- **Placeholder content**: TODO or template prose remains in runtime instructions.
- **Broken examples**: Included scripts or commands fail syntax or focused execution checks.
Every qualitative finding must cite a file and line and state the behavioral impact. Do not recommend adding content solely to increase line count.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |