Skip to content

JasonColapietro/suede-creator-skills / skills/suede-ab-testing

suede-ab-testing

Verified

Suede A/B Testing Skills. Use when asked to "design an A/B test", compare variants, assess reliability, or build an experiment backlog: hypothesis, power, duration, guardrails and a predeclared decision rule. NOT FOR: instrumentation (suede-analytics), funnel diagnosis (suede-site-alchemy), variant copy (suede-copy), or channel incrementality (suede-attribution).

FileKindSize
CARD.mdmarkdown4.2 KB
SKILL.mdskill md14.6 KB
agents/openai.yamlconfig586 B
evals/evals.jsonconfig7.7 KB
references/sample-size-guide.mdmarkdown7.4 KB
references/test-templates.mdmarkdown6.3 KB
references/worked-contract.mdmarkdown1.7 KB
# Skill Card — Suede A/B Testing Skills

<!-- Generated by scripts/build-skill-cards.mjs — do not hand-edit. -->
<!-- Regenerate with: npm run build:cards -->

Release record for the `suede-ab-testing` skill, following the NVIDIA skill-card template (<https://docs.nvidia.com/skills/skill-cards>). It tells a reviewer what the skill does, who owns it, what it needs, what could go wrong, and what evidence backs the release — without requiring them to open the source first.

## Description

Suede A/B Testing Skills.

Status: production. Ships in the `suede-skills` plugin (the full pack) at release 0.19.4; loads as a Claude Code / Codex agent skill from this directory's [SKILL.md](./SKILL.md).

## Owner

Jason Colapietro, Suede (<https://github.com/JasonColapietro>). Security contact: `info@suedeai.ai` per [SECURITY.md](../../SECURITY.md).

## License / Terms of Use

MIT ([LICENSE](../../LICENSE)). The pack's combined license expression is `MIT AND BSD-3-Clause`; this skill bundles no third-party licensed material of its own.

## Use Case

Target users: developers and creators running the skill inside a Claude Code or Codex CLI session.

Use when asked to \"design an A/B test\", compare variants, assess reliability, or build an experiment backlog: hypothesis, power, duration, guardrails and a predeclared decision rule.

Out of scope — instrumentation (suede-analytics), funnel diagnosis (suede-site-alchemy), variant copy (suede-copy), or channel incrementality (suede-attribution).

## Deployment Geography

Global. The skill is a prompt-and-script package that runs locally inside the invoking agent session; it pins no region-specific service of its own.

## Requirements / Dependencies

- A Claude Code or Codex CLI session with the `suede-skills` plugin installed (install options: <https://skills.suedeai.ai/>).
- Bundled files loaded relative to this directory: `agents/` (1 file), `references/` (3 files).
- Credentials: none are bundled or required by the skill files. Any tool or API credentials come from the host session; never paste credentials into skill files, prompts, or outputs.

## Known Risks and Mitigations

- Risk: an agent treats a quality gate as autonomous authority. Mitigation: every gate in the pack is advisory — it changes what is reported, never what the user decided; only extreme-risk findings (data loss, credential exposure, legal/rights violations, payment mistakes, irreversible public damage) pause for the user's explicit choice.
- Risk: a skill instruction is used to act outside its mandate. Mitigation: the hard limits in the skill body's "Boundaries" section, quoted below.

From "Boundaries":

- Do not claim a winner before the predeclared sample, duration, and decision rule are satisfied.
- Do not alter production traffic allocation, experiment settings, or analytics without explicit authorization and a rollback path.
- Do not publish results without reporting uncertainty, guardrail movement, exclusions, and stopped-early status.
- Do not decide that statistical significance equals business value; compare the effect with the minimum useful lift.

## References

- Skill source: [`skills/suede-ab-testing/SKILL.md`](./SKILL.md)
- Rendered reference page: <https://skills.suedeai.ai/skills/suede-ab-testing.html>
- Security policy and reviewed scanner exceptions: [SECURITY.md](../../SECURITY.md) and [`.plugin-scanner.toml`](../../.plugin-scanner.toml) at the repo root

## Skill Output

Markdown analysis and recommendations returned in the agent's response. The skill publishes, posts, and sends nothing without the user's explicit authorization; delivery decisions stay with the user.

## Skill Version

0.19.4 — the pack is single-versioned, so every skill releases together; see [VERSION](../../VERSION) and [CITATION.cff](../../CITATION.cff) for the release identifier this card describes.

## Ethical Considerations

- The skill produces recommendations for a human decision-maker. Publishing, sending, payment, and rights decisions stay with the user.
- Its gates require verifiable claims and honest reporting; do not use the skill to fabricate claims, evidence, metrics, or attribution.
- Report suspected misuse or a security concern privately per [SECURITY.md](../../SECURITY.md); do not open a public issue for it.
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall