diegosouzapw/awesome-omni-skill / skills/ai-agents/agent-evaluation-cdalsoniii
Agent Evaluation
VerifiedEvaluate agent performance using a structured scoring rubric
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 2.0 KB |
| metadata.json | config | 1.3 KB |
SKILL.md
View on GitHub---
name: Agent Evaluation
description: Evaluate agent performance using a structured scoring rubric
category: ai-agents
version: 1.0.0
triggers:
- agent-config-change
- evaluation-cadence
- performance-review
globs: "**/agents/**,**/skills/**"
---
# Agent Evaluation Skill
Evaluate agent performance using a structured scoring rubric.
## Trigger Conditions
- Agent configuration change
- Evaluation cadence (monthly)
- User invokes with "evaluate agents" or "agent scorecard"
## Input Contract
- **Required:** Agent(s) to evaluate
- **Required:** Evaluation criteria or rubric
- **Optional:** Baseline scores from prior evaluation
## Output Contract
- Evaluation scorecard (500-point rubric)
- Per-dimension scores and findings
- Improvement recommendations
- Comparison against baseline
## Tool Permissions
- **Read:** Agent configs, agent output logs, telemetry
- **Write:** Evaluation reports
- **Search:** Agent invocation history
## Execution Steps
1. Load evaluation rubric (architecture 100, security 100, ops 100, testing 100, docs 100)
2. For each agent, review recent outputs and effectiveness
3. Score each dimension with evidence
4. Compare against baseline scores
5. Identify improvement opportunities
6. Generate scorecard and recommendations
## Success Criteria
- All dimensions scored with evidence
- Comparison against prior evaluation
- Top 3 improvement recommendations per agent
- Overall portfolio health assessment
## Escalation Rules
- Escalate if any agent scores below 50% on any dimension
- Escalate if agent evaluation reveals conflicting outputs between agents
## Example Invocations
**Input:** "Evaluate the security-specialist agent effectiveness"
**Output:** Scorecard: Security (85/100), Architecture (70/100), Ops (75/100), Testing (60/100), Docs (80/100). Total: 370/500. Findings: strong CVE detection but weak test coverage recommendations, documentation quality high but missing escalation follow-through. Top improvement: integrate with testing-specialist for security test gap analysis.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |