hoavdc/CodexKit / skills/codexkit-a-b-test-planner
codexkit-a-b-test-planner
VerifiedDesign rigorous A/B test plans with hypothesis, sample size calculation, Minimum Detectable Effect (MDE), randomization strategy, and decision rules. Includes guardrail metrics and rollout playbook. Use when planning product experiments, conversion optimization, or data-driven feature decisions.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 5.5 KB |
| agents/openai.yaml | config | 165 B |
| examples/common-mistakes.md | markdown | 794 B |
| examples/good-output.md | markdown | 870 B |
| verification/checklist.md | markdown | 1.4 KB |
examples/good-output.md
View on GitHub# Good Output Example — A/B Test Planner
> This example demonstrates what well-executed a/b test planner output looks like.
## Scenario
A/B test for a simplified checkout flow on an e-commerce site
## Why It Works
1. Hypothesis: 'Reducing checkout steps from 4 to 2 will increase completion rate by ≥3pp'
2. Power: 'Baseline: 28% completion. MDE: 3pp. α=0.05, β=0.20 → n=3,400/variant → 14 days at current traffic'
3. Guardrails: 'Average order value must not drop >2%. Support ticket rate must not increase >10%.'
4. Decision: 'Ship if primary metric is significant AND no guardrail violated. Hold if borderline for 7 more days.'
## Key Patterns to Replicate
- Hypothesis is falsifiable and specific
- Duration is calculated, not arbitrary ('let's run it for 2 weeks')
- Success criteria include both 'what we want' and 'what we can't afford to break'
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 321k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 321k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 321k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 197k | Updated | Claude CodeCodex | Verified | Install |