hoavdc/CodexKit / skills/codexkit-a-b-test-planner
codexkit-a-b-test-planner
VerifiedDesign rigorous A/B test plans with hypothesis, sample size calculation, Minimum Detectable Effect (MDE), randomization strategy, and decision rules. Includes guardrail metrics and rollout playbook. Use when planning product experiments, conversion optimization, or data-driven feature decisions.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 5.5 KB |
| agents/openai.yaml | config | 165 B |
| examples/common-mistakes.md | markdown | 794 B |
| examples/good-output.md | markdown | 870 B |
| verification/checklist.md | markdown | 1.4 KB |
examples/common-mistakes.md
View on GitHub# Common Mistakes — A/B Test Planner
> Anti-patterns to watch for when using the A/B Test Planner skill.
## 1. Peeking and stopping early
**What goes wrong:** Checking results daily and stopping when p<0.05 — massively inflates false positives
**How to fix it:** Pre-commit to test duration. If you must peek, use sequential testing methods.
## 2. Testing too many changes at once
**What goes wrong:** New layout + new copy + new colors — can't attribute the effect
**How to fix it:** One change per test. If testing multiple factors, use a proper factorial design.
## 3. Ignoring guardrails
**What goes wrong:** Conversion up 5% but support tickets up 40% — net negative experience
**How to fix it:** Define guardrail metrics upfront. A 'win' must not violate any guardrail.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 197k | Updated | Claude CodeCodex | Verified | Install |