Skip to content

hoavdc/CodexKit / skills/codexkit-a-b-test-planner

codexkit-a-b-test-planner

Verified

Design rigorous A/B test plans with hypothesis, sample size calculation, Minimum Detectable Effect (MDE), randomization strategy, and decision rules. Includes guardrail metrics and rollout playbook. Use when planning product experiments, conversion optimization, or data-driven feature decisions.

FileKindSize
SKILL.mdskill md5.5 KB
agents/openai.yamlconfig165 B
examples/common-mistakes.mdmarkdown794 B
examples/good-output.mdmarkdown870 B
verification/checklist.mdmarkdown1.4 KB

examples/good-output.md

View on GitHub
# Good Output Example — A/B Test Planner

> This example demonstrates what well-executed a/b test planner output looks like.

## Scenario

A/B test for a simplified checkout flow on an e-commerce site

## Why It Works

1. Hypothesis: 'Reducing checkout steps from 4 to 2 will increase completion rate by ≥3pp'
2. Power: 'Baseline: 28% completion. MDE: 3pp. α=0.05, β=0.20 → n=3,400/variant → 14 days at current traffic'
3. Guardrails: 'Average order value must not drop >2%. Support ticket rate must not increase >10%.'
4. Decision: 'Ship if primary metric is significant AND no guardrail violated. Hold if borderline for 7 more days.'

## Key Patterns to Replicate

- Hypothesis is falsifiable and specific
- Duration is calculated, not arbitrary ('let's run it for 2 weeks')
- Success criteria include both 'what we want' and 'what we can't afford to break'
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 321kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 321kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 321kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 197kUpdated Claude CodeCodexVerifiedInstall