dvy1987/agent-loom / .agents/skills/eval-output
eval-output
VerifiedOrchestrator for the eval-output skill suite — evaluate LLM and agent outputs for quality, accuracy, helpfulness, and safety using structured rubrics and LLM-as-judge techniques. Load when the user says "evaluate this output", "score this response", "run an eval", "LLM as judge", "evaluate agent output", "how good is this response", "rate this answer", "eval this", or provides an LLM output that should be assessed for quality. Single entry point for all output evaluation workflows.
| File | Kind | Size |
|---|---|---|
| README.md | markdown | 765 B |
| SKILL.md | skill md | 7.6 KB |
| references/examples.md | markdown | 1.9 KB |
references/examples.md
View on GitHub# Eval Output — Full Worked Examples
Skill: `eval-output` | Enriched from SKILL.md (improve-skills pass, SKIP_RESEARCH).
## Example 1 — Documented workflow
**Input:** Evaluate this response my agent gave about database indexing
**Output:**
```
=== Eval Output Report ===
Target: Agent response on database indexing
Eval type: Needs rubric first
No rubric found for this task. Routing to eval-rubric-design to create one.
[Invokes eval-rubric-design → user approves rubric → invokes eval-judge with rubric]
=== Evaluation ===
[Structured scores per dimension with justifications]
=== Summary ===
Overall: 3 of 5 dimensions scored 4+/5. Accuracy strong, completeness weak.
Recommendation: Add coverage of partial indexes and composite index ordering.
```
## Example 2 — Step-by-step execution
**Input:** "Run `eval-output` on [concrete task]"
**Agent actions:**
1. Accept Input
2. Classify Evaluation Need
3. Route to Sub-Skill
4. Unified Report
**Impact Report shape:**
```
=== Eval Output Report ===
Target: [what was evaluated — output type, task, model]
Eval type: [rubric-design / direct-scoring / pairwise / pipeline-design]
=== Evaluation ===
[Sub-skill specific output]
=== Summary ===
[Key findings, recommendations, next steps]
```
## Example 3 — Gotcha application
**Input:** Task hits a non-obvious edge case
**Apply:**
- An output that "sounds good" can still fail on accuracy, safety, or completeness — never skip structured evaluation because the output reads well.
- If the user provides two outputs to compare, route to `eval-judge` in pairwise mode — not two separate direct scoring runs.
- Rubrics drift over time as tasks and models evolve. Recommend periodic rubric review when eval results change unexpectedly.
- Self-evaluation (model judging its own output) has known self-enhancement bias. Recommend a different model for judging when possible.
---
See `SKILL.md` for hard rules and verification checklist.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |