Skip to content

JasonColapietro/suede-creator-skills / skills/suede-ab-testing

suede-ab-testing

Verified

Suede A/B Testing Skills. Use when asked to "design an A/B test", compare variants, assess reliability, or build an experiment backlog: hypothesis, power, duration, guardrails and a predeclared decision rule. NOT FOR: instrumentation (suede-analytics), funnel diagnosis (suede-site-alchemy), variant copy (suede-copy), or channel incrementality (suede-attribution).

FileKindSize
CARD.mdmarkdown4.2 KB
SKILL.mdskill md14.6 KB
agents/openai.yamlconfig586 B
evals/evals.jsonconfig7.7 KB
references/sample-size-guide.mdmarkdown7.4 KB
references/test-templates.mdmarkdown6.3 KB
references/worked-contract.mdmarkdown1.7 KB

references/worked-contract.md

View on GitHub
# Worked delivery contract

This synthetic scenario demonstrates the expected artifact shape; its numbers and outcomes are example inputs, not measured customer results.

## Input

A checkout has 10,000 eligible sessions per week, a 4% purchase baseline and a proposed change to shipping-price visibility. The owner proposes a 10% relative improvement, not ten percentage points.

## Expected output

Write the hypothesis, randomization unit, primary purchase metric and denominators, 4.0% control and 4.4% alternative rates, chosen alpha/power and sample-size calculation inputs. Predeclare duration, a refund guardrail, assignment checks and the one-time or sequential decision rule. Return a plan; do not claim a winner before observations exist.

## Failure case

The export reports users in one arm and sessions in the other. Mark the readout non-comparable, retain raw counts and request the common unit/window. Do not compute significance on incompatible denominators.

## Validation

Recalculate absolute versus relative uplift and sample-size inputs independently. Ensure every arm uses the same eligibility/window and the guardrail stop is distinguished from a winning result.

## Coverage and routing

Suede experimentation discipline: hypotheses, sample size and power, test duration, guardrail metrics, significance and readouts, sequential designs, and an ICE-ranked experiment backlog with a reusable playbook. Use when comparing variants, deciding whether a result is reliable, or building an experiment program and cadence. NOT FOR: analytics instrumentation (use suede-analytics), post-click conversion diagnosis (use suede-site-alchemy), writing the variant copy (use suede-copy), or channel-level incrementality and geo holdouts (use suede-attribution).
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 198kUpdated Claude CodeCodexVerifiedInstall