JasonColapietro/suede-creator-skills / skills/suede-ab-testing
suede-ab-testing
VerifiedSuede A/B Testing Skills. Use when asked to "design an A/B test", compare variants, assess reliability, or build an experiment backlog: hypothesis, power, duration, guardrails and a predeclared decision rule. NOT FOR: instrumentation (suede-analytics), funnel diagnosis (suede-site-alchemy), variant copy (suede-copy), or channel incrementality (suede-attribution).
| File | Kind | Size |
|---|---|---|
| CARD.md | markdown | 4.2 KB |
| SKILL.md | skill md | 14.6 KB |
| agents/openai.yaml | config | 586 B |
| evals/evals.json | config | 7.7 KB |
| references/sample-size-guide.md | markdown | 7.4 KB |
| references/test-templates.md | markdown | 6.3 KB |
| references/worked-contract.md | markdown | 1.7 KB |
references/worked-contract.md
View on GitHub# Worked delivery contract
This synthetic scenario demonstrates the expected artifact shape; its numbers and outcomes are example inputs, not measured customer results.
## Input
A checkout has 10,000 eligible sessions per week, a 4% purchase baseline and a proposed change to shipping-price visibility. The owner proposes a 10% relative improvement, not ten percentage points.
## Expected output
Write the hypothesis, randomization unit, primary purchase metric and denominators, 4.0% control and 4.4% alternative rates, chosen alpha/power and sample-size calculation inputs. Predeclare duration, a refund guardrail, assignment checks and the one-time or sequential decision rule. Return a plan; do not claim a winner before observations exist.
## Failure case
The export reports users in one arm and sessions in the other. Mark the readout non-comparable, retain raw counts and request the common unit/window. Do not compute significance on incompatible denominators.
## Validation
Recalculate absolute versus relative uplift and sample-size inputs independently. Ensure every arm uses the same eligibility/window and the guardrail stop is distinguished from a winning result.
## Coverage and routing
Suede experimentation discipline: hypotheses, sample size and power, test duration, guardrail metrics, significance and readouts, sequential designs, and an ICE-ranked experiment backlog with a reusable playbook. Use when comparing variants, deciding whether a result is reliable, or building an experiment program and cadence. NOT FOR: analytics instrumentation (use suede-analytics), post-click conversion diagnosis (use suede-site-alchemy), writing the variant copy (use suede-copy), or channel-level incrementality and geo holdouts (use suede-attribution).
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |