Skip to content

rampstackco/claude-skills / skills/experiment-design

experiment-design

Flagged

A discipline for designing experiments (A/B tests, multivariate, holdouts) so the results actually answer the question you asked. Hypothesis writing, sample size, duration, segment analysis, running discipline, matching a result to a pre-committed decision rule, and the common failure modes that produce confidently wrong shipping decisions. Use this skill whenever the user is planning a test that has not run yet: framing a hypothesis, sizing the sample, setting duration, choosing guardrails, or deciding whether something is worth testing at all. Triggers on design an experiment, experiment plan, A/B test, split test, multivariate test, holdout, experiment hypothesis, sample size, minimum detectable effect, MDE, test duration, guardrail metric, no peeking, pre-committed decision rule, is this worth testing. Use `experimentation-analytics` instead when the test has already run and the question is how to read the result panel.

Security report

Flagged

Static analysis of one exact version. It reduces risk and shows evidence; it does not certify that a skill is safe.

Commit
9d281db6e7a5c4a0084ba89c91633a07f5f7e42b
Content hash
b7be671c0cc4fc8db84fd51f234b2f49faa4d592a4c2a8586b61f2ac82495936
Scanned
2026-11-17
Scanner version
skillxone-scan-1.0.0
Policy version
0.1-draft
Decision
1 open critical or high finding.
Verification expires
Not applicable

Findings (1 high)

  • highPI-DISABLE-SAFEGUARDSSKILL.md:13

    The text tells the agent to disable safety checks, permission prompts or other safeguards.

    The default state of experimentation in most companies is sloppy. PMs run tests against vague hypotheses, look at results too early, ignore guardrails, stratify into noise, and ship features whose lift is mostly measurement error. The cost…

    Suggested fix: Remove the instruction. If the task genuinely needs elevated permissions, say so plainly and let the user decide.

    Report as false positive

Checks run

  • prompt-injection
  • dangerous-commands
  • credentials
  • network
  • file-access
  • integrity

Checks skipped

  • dependencies: No dependency manifest in the skill directory.
  • gitleaks: gitleaks is not installed on this host.
finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 322kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 322kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 322kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 197kUpdated Claude CodeCodexVerifiedInstall