Skip to content

dvy1987/agent-loom / .agents/skills/eval-pipeline

eval-pipeline

Verified

Design automated evaluation pipelines for LLM and agent systems — combining deterministic checks, statistical metrics, and LLM-as-judge scoring into repeatable, CI-integrated eval suites. Load when the user asks to set up automated evals, design an eval pipeline, integrate evals into CI/CD, create an eval suite, do eval-driven development, or says "automate my evals", "CI eval integration", "evaluation pipeline", "continuous evaluation", "monitoring eval quality", "set up regression testing for my agent". Sub-skill of eval-output orchestrator.

FileKindSize
README.mdmarkdown625 B
SKILL.mdskill md11.0 KB
references/examples.mdmarkdown2.2 KB
references/harness-regression.mdmarkdown2.2 KB
# eval-pipeline

> Design automated evaluation pipelines for LLM and agent systems — deterministic checks, statistical metrics, and LLM-as-judge scoring with CI/CD integration.

## Install

```bash
npx skills eval-pipeline -a codex      # Codex
npx skills eval-pipeline -a replit     # Replit
cp -r eval-pipeline/ .agents/skills/   # Universal
```

## Usage

Trigger: "set up automated evals", "design an eval pipeline", "CI eval integration", "eval-driven development"

## Platforms

Works with: Codex, Ampcode, Claude Code, Warp, Gemini CLI, GitHub Copilot, Cursor, VS Code, Replit, Factory.ai, Bolt.new

## License

MIT
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall