a5c-ai/babysitter / library/specializations/data-engineering-analytics/skills/ab-test-statistical-analyzer
A/B Test Statistical Analyzer
VerifiedPerforms statistical analysis for A/B testing experiments
A/B Test Statistical Analyzer
Overview
Performs statistical analysis for A/B testing experiments. This skill provides rigorous statistical methods to determine experiment validity and significance.
Capabilities
- Sample size calculation
- Statistical significance testing
- Bayesian analysis
- Sequential testing
- Multi-armed bandit analysis
- Segment analysis
- Novelty/primacy effect detection
- SRM (Sample Ratio Mismatch) detection
- Confidence interval calculation
- Power analysis
Input Schema
{
"experimentData": {
"control": "object",
"variants": ["object"]
},
"metrics": [{
"name": "string",
"type": "conversion|continuous|ratio"
}],
"analysisType": "frequentist|bayesian|sequential"
}
Output Schema
{
"results": [{
"metric": "string",
"controlValue": "number",
"variantValues": ["number"],
"pValue": "number",
"confidenceInterval": "object",
"significant": "boolean"
}],
"srmCheck": "object",
"recommendation": "string"
}
Target Processes
- A/B Testing Pipeline
- Feature Store Setup
Usage Guidelines
- Provide complete experiment data for control and variants
- Define metrics with appropriate types
- Select analysis methodology based on requirements
- Review SRM checks before interpreting results
Best Practices
- Always check for sample ratio mismatch before analysis
- Use appropriate statistical tests for metric types
- Consider practical significance alongside statistical significance
- Account for multiple comparison corrections
- Document assumptions and limitations
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 197k | Updated | Claude CodeCodex | Verified | Install |