Skip to content

JasonColapietro/suede-creator-skills / skills/suede-ai-eval

suede-ai-eval

Verified

Suede Eval Skills. Use when asked to "write evals for this AI feature" or assess LLM, RAG, agent, classifier or generated-media behavior: AI-SPEC, failure rubric, executable cases, coverage and acceptance evidence. NOT FOR: implementation review (suede-code), wiring existing evals into CI (suede-ci-gate), or coordinating builder lanes (suede-agent-teams).

FileKindSize
CARD.mdmarkdown4.2 KB
SKILL.mdskill md12.8 KB
agents/openai.yamlconfig306 B
references/eval-case-design.mdmarkdown10.5 KB
references/worked-contract.mdmarkdown2.0 KB
# Skill Card — Suede Eval Skills

<!-- Generated by scripts/build-skill-cards.mjs — do not hand-edit. -->
<!-- Regenerate with: npm run build:cards -->

Release record for the `suede-ai-eval` skill, following the NVIDIA skill-card template (<https://docs.nvidia.com/skills/skill-cards>). It tells a reviewer what the skill does, who owns it, what it needs, what could go wrong, and what evidence backs the release — without requiring them to open the source first.

## Description

Suede Eval Skills.

Status: production. Ships in the `suede-skills` plugin (the full pack) at release 0.19.4; loads as a Claude Code / Codex agent skill from this directory's [SKILL.md](./SKILL.md).

## Owner

Jason Colapietro, Suede (<https://github.com/JasonColapietro>). Security contact: `info@suedeai.ai` per [SECURITY.md](../../SECURITY.md).

## License / Terms of Use

MIT ([LICENSE](../../LICENSE)). The pack's combined license expression is `MIT AND BSD-3-Clause`; this skill bundles no third-party licensed material of its own.

## Use Case

Target users: developers and creators running the skill inside a Claude Code or Codex CLI session.

Use when asked to \"write evals for this AI feature\" or assess LLM, RAG, agent, classifier or generated-media behavior: AI-SPEC, failure rubric, executable cases, coverage and acceptance evidence.

Out of scope — implementation review (suede-code), wiring existing evals into CI (suede-ci-gate), or coordinating builder lanes (suede-agent-teams).

## Deployment Geography

Global. The skill is a prompt-and-script package that runs locally inside the invoking agent session; it pins no region-specific service of its own.

## Requirements / Dependencies

- A Claude Code or Codex CLI session with the `suede-skills` plugin installed (install options: <https://skills.suedeai.ai/>).
- Bundled files loaded relative to this directory: `agents/` (1 file), `references/` (2 files).
- Credentials: none are bundled or required by the skill files. Any tool or API credentials come from the host session; never paste credentials into skill files, prompts, or outputs.

## Known Risks and Mitigations

- Risk: an agent treats a quality gate as autonomous authority. Mitigation: every gate in the pack is advisory — it changes what is reported, never what the user decided; only extreme-risk findings (data loss, credential exposure, legal/rights violations, payment mistakes, irreversible public damage) pause for the user's explicit choice.
- Risk: a skill instruction is used to act outside its mandate. Mitigation: the hard limits in the skill body's "Boundaries" section, quoted below.

From "Boundaries":

- Do not claim legal, rights, licensing, medical, financial, or compliance clearance.
- Do not invent private datasets, logs, scores, or customer outcomes.
- Do not upload data, call private services, or run destructive workflows unless the user explicitly asks and the repo/tooling supports it.
- Do not treat a model's self-judgment as sufficient evidence.
- Do not mark eval coverage complete when only prompt review or happy-path manual testing exists.

## References

- Skill source: [`skills/suede-ai-eval/SKILL.md`](./SKILL.md)
- Rendered reference page: <https://skills.suedeai.ai/skills/suede-ai-eval.html>
- Security policy and reviewed scanner exceptions: [SECURITY.md](../../SECURITY.md) and [`.plugin-scanner.toml`](../../.plugin-scanner.toml) at the repo root

## Skill Output

Structured Markdown returned in the agent's response, shaped by the output contract defined in the skill body: "Output". The skill publishes, posts, and sends nothing without the user's explicit authorization; delivery decisions stay with the user.

## Skill Version

0.19.4 — the pack is single-versioned, so every skill releases together; see [VERSION](../../VERSION) and [CITATION.cff](../../CITATION.cff) for the release identifier this card describes.

## Ethical Considerations

- The skill produces recommendations for a human decision-maker. Publishing, sending, payment, and rights decisions stay with the user.
- Its gates require verifiable claims and honest reporting; do not use the skill to fabricate claims, evidence, metrics, or attribution.
- Report suspected misuse or a security concern privately per [SECURITY.md](../../SECURITY.md); do not open a public issue for it.
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 323kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 323kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 323kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall