JasonColapietro/suede-creator-skills / skills/suede-ai-eval
suede-ai-eval
VerifiedSuede Eval Skills. Use when asked to "write evals for this AI feature" or assess LLM, RAG, agent, classifier or generated-media behavior: AI-SPEC, failure rubric, executable cases, coverage and acceptance evidence. NOT FOR: implementation review (suede-code), wiring existing evals into CI (suede-ci-gate), or coordinating builder lanes (suede-agent-teams).
| File | Kind | Size |
|---|---|---|
| CARD.md | markdown | 4.2 KB |
| SKILL.md | skill md | 12.8 KB |
| agents/openai.yaml | config | 306 B |
| references/eval-case-design.md | markdown | 10.5 KB |
| references/worked-contract.md | markdown | 2.0 KB |
CARD.md
View on GitHub# Skill Card — Suede Eval Skills
<!-- Generated by scripts/build-skill-cards.mjs — do not hand-edit. -->
<!-- Regenerate with: npm run build:cards -->
Release record for the `suede-ai-eval` skill, following the NVIDIA skill-card template (<https://docs.nvidia.com/skills/skill-cards>). It tells a reviewer what the skill does, who owns it, what it needs, what could go wrong, and what evidence backs the release — without requiring them to open the source first.
## Description
Suede Eval Skills.
Status: production. Ships in the `suede-skills` plugin (the full pack) at release 0.19.4; loads as a Claude Code / Codex agent skill from this directory's [SKILL.md](./SKILL.md).
## Owner
Jason Colapietro, Suede (<https://github.com/JasonColapietro>). Security contact: `info@suedeai.ai` per [SECURITY.md](../../SECURITY.md).
## License / Terms of Use
MIT ([LICENSE](../../LICENSE)). The pack's combined license expression is `MIT AND BSD-3-Clause`; this skill bundles no third-party licensed material of its own.
## Use Case
Target users: developers and creators running the skill inside a Claude Code or Codex CLI session.
Use when asked to \"write evals for this AI feature\" or assess LLM, RAG, agent, classifier or generated-media behavior: AI-SPEC, failure rubric, executable cases, coverage and acceptance evidence.
Out of scope — implementation review (suede-code), wiring existing evals into CI (suede-ci-gate), or coordinating builder lanes (suede-agent-teams).
## Deployment Geography
Global. The skill is a prompt-and-script package that runs locally inside the invoking agent session; it pins no region-specific service of its own.
## Requirements / Dependencies
- A Claude Code or Codex CLI session with the `suede-skills` plugin installed (install options: <https://skills.suedeai.ai/>).
- Bundled files loaded relative to this directory: `agents/` (1 file), `references/` (2 files).
- Credentials: none are bundled or required by the skill files. Any tool or API credentials come from the host session; never paste credentials into skill files, prompts, or outputs.
## Known Risks and Mitigations
- Risk: an agent treats a quality gate as autonomous authority. Mitigation: every gate in the pack is advisory — it changes what is reported, never what the user decided; only extreme-risk findings (data loss, credential exposure, legal/rights violations, payment mistakes, irreversible public damage) pause for the user's explicit choice.
- Risk: a skill instruction is used to act outside its mandate. Mitigation: the hard limits in the skill body's "Boundaries" section, quoted below.
From "Boundaries":
- Do not claim legal, rights, licensing, medical, financial, or compliance clearance.
- Do not invent private datasets, logs, scores, or customer outcomes.
- Do not upload data, call private services, or run destructive workflows unless the user explicitly asks and the repo/tooling supports it.
- Do not treat a model's self-judgment as sufficient evidence.
- Do not mark eval coverage complete when only prompt review or happy-path manual testing exists.
## References
- Skill source: [`skills/suede-ai-eval/SKILL.md`](./SKILL.md)
- Rendered reference page: <https://skills.suedeai.ai/skills/suede-ai-eval.html>
- Security policy and reviewed scanner exceptions: [SECURITY.md](../../SECURITY.md) and [`.plugin-scanner.toml`](../../.plugin-scanner.toml) at the repo root
## Skill Output
Structured Markdown returned in the agent's response, shaped by the output contract defined in the skill body: "Output". The skill publishes, posts, and sends nothing without the user's explicit authorization; delivery decisions stay with the user.
## Skill Version
0.19.4 — the pack is single-versioned, so every skill releases together; see [VERSION](../../VERSION) and [CITATION.cff](../../CITATION.cff) for the release identifier this card describes.
## Ethical Considerations
- The skill produces recommendations for a human decision-maker. Publishing, sending, payment, and rights decisions stay with the user.
- Its gates require verifiable claims and honest reporting; do not use the skill to fabricate claims, evidence, metrics, or attribution.
- Report suspected misuse or a security concern privately per [SECURITY.md](../../SECURITY.md); do not open a public issue for it.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 323k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 323k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 323k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |