JPeetz/agent-skills / skills/qa-testing/agent-evaluation
agent-evaluation
UnverifiedUse this skill when building systematic evaluation for LLM agents and agent outputs — defining quality dimensions, curating golden datasets, running LLM-as-judge, pairwise and rubric-based evaluations, tracking regressions, and gating CI on quality. Covers promptfoo, DeepEval, LangSmith, Inspect, and lm-evaluation-harness, with judge calibration, cost/latency budgets, and eval-driven development for prompt, model, and tool changes. Not for deterministic unit tests of application code (see test-driven-development), tracing/dashboards only (see llm-observability), or evaluator-optimizer architecture (see agentic-eval).
| Commit | Indexed | Files | Content hash |
|---|---|---|---|
a70437dcurrent | 2026-11-17 | 8 · 57.3 KB | 7acbc98736b9279d… |
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |