K-Dense-AI/scientific-agent-skills / skills/experimental-design
experimental-design
VerifiedDesigns experiments and studies BEFORE data is collected — choosing a design, randomizing, blocking, and laying out treatment combinations so results are interpretable. Use whenever someone is planning a study, asks how to assign subjects/samples to groups, mentions randomization, blocking, stratification, controls, factorial or fractional-factorial designs, design of experiments (DOE), screening many factors, response-surface optimization, crossover or repeated-measures or split-plot designs, cluster/group randomization, Latin squares, plate layouts, batch/run-order effects, replication vs. pseudoreplication, or sequential/adaptive/group-sequential designs. Trigger even for informal phrasings like "how should I set up this experiment", "how do I avoid confounding", "what's the best way to test these 6 factors", or "assign these mice to conditions". For computing the sample size or power once the design is chosen, use statistical-power; for analyzing data already collected, use statistical-analysis.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 15.9 KB |
| references/design_types.md | markdown | 7.8 KB |
| references/factorial_and_doe.md | markdown | 8.9 KB |
| references/randomization_and_blocking.md | markdown | 7.6 KB |
| references/sequential_and_adaptive.md | markdown | 6.4 KB |
| scripts/doe_designs.py | script | 9.4 KB |
| scripts/randomization.py | script | 9.4 KB |
references/factorial_and_doe.md
View on GitHub# Factorial and Design-of-Experiments (DOE)
When several factors might affect a response, testing them **one factor at a time
(OFAT)** is both wasteful and blind to interactions. Factorial designs vary factors
*together*, so the appropriate full design separates main effects and interactions;
reduced designs trade some of that information for fewer runs. This file covers the family of DOE designs and the
concepts (resolution, aliasing) needed to read them. Generate them with
`scripts/doe_designs.py`.
## Table of contents
- [Why factorial beats OFAT](#why-factorial-beats-ofat)
- [Full factorial (2^k)](#full-factorial)
- [Fractional factorial (2^(k-p))](#fractional-factorial)
- [Resolution and aliasing](#resolution-and-aliasing)
- [Screening designs (Plackett-Burman)](#screening-designs)
- [Response-surface designs](#response-surface-designs)
- [Space-filling designs](#space-filling-designs)
- [Choosing a design](#choosing-a-design)
## Why factorial beats OFAT
Vary one factor while holding others fixed and you (1) spend runs inefficiently and
(2) can never see **interactions** — cases where the effect of A depends on the level
of B, which are the rule, not the exception, in real systems. A factorial varies all
factors simultaneously across runs; each effect is estimated using *all* the data, so
a 2^k factorial is more precise than k separate OFAT studies of the same size.
## Full factorial
A **2^k** design runs every combination of k factors at two levels (low/high, coded
−1/+1). It estimates all k main effects and all 2^k − k − 1 interactions. With one observation per combination, the
model containing every effect plus an intercept is saturated: it has zero residual
degrees of freedom. Replicate independently or pre-specify a defensible reduced
model before claiming standard errors or significance.
- Runs = 2^k: 8 for 3 factors, 16 for 4, 32 for 5. Practical to ~5 factors.
- Use when you have a handful of factors and want a full picture including
interactions.
- `two_level_factorial({"temp": (20,60), "conc": (1,10), "pH": (6,8)})` → 8 runs.
- For factors with more than two levels, use `full_factorial` with explicit level
lists (runs = product of level counts — grows fast).
Add **center points** (all factors at their midpoint) to a two-level design to get a
cheap check for curvature: if the center response departs from the factorial average,
a linear model is inadequate and you need a response-surface design.
## Fractional factorial
When k is large, 2^k is too many runs — but most high-order interactions are
negligible (the *sparsity-of-effects* principle). A **2^(k−p)** fractional factorial
runs a carefully chosen fraction (1/2, 1/4, ...) of the full design, trading the
ability to estimate some interactions for far fewer runs.
- `fractional_factorial(factors, generator="a b c abc")` builds a half-fraction of 4
factors in 8 runs. The generator string (Yates notation) assigns each factor to a
column; a multi-letter token aliases that factor with an interaction.
- The price is **aliasing**: some effects become indistinguishable. You must know
which.
## Resolution and aliasing
**Aliasing** (confounding) means two effects are estimated by the same contrast — the
data cannot separate them. Which effects are aliased is summarized by the design's
**resolution**:
| Resolution | Aliasing | Interpretation |
|------------|----------|----------------|
| **III** | main effects aliased with 2-factor interactions | Screening only; a "significant" main effect might be an interaction |
| **IV** | main effects clear of 2FI, but 2FIs aliased with each other | Main effects require negligible aliased 3FI+ |
| **V** | main effects and 2FIs all clear of each other (aliased with 3FI+) | Main/2FI separation assumes negligible aliased higher effects |
Always state the resolution and inspect the alias structure before interpreting a
fractional design. Concluding "factor C has no effect" is unsafe if C is aliased with
a real interaction (it could cancel out). When in doubt, choose a higher-resolution
generator (more runs) or add runs to **de-alias** (fold-over / augment the design).
## Screening designs
When the goal is to **find the vital few** factors out of many (5, 10, 20+), use a
screening design that estimates main effects only, as cheaply as possible:
- **Plackett-Burman** (`plackett_burman`): runs = the next multiple of 4 above k
(e.g. 12 runs for up to 11 factors). Resolution III — two-factor interactions are
often confounded with main effects. Use for triage: run it, keep the few
factors with large effects, then study those with a full or higher-resolution
factorial.
- Resolution III fractional factorials serve the same purpose.
The current `pydoe==1.5.0` implementation supports run counts in the families
`2^a`, `12*2^a`, and `20*2^a`. It can reject a requested factor count whose next
multiple of four is unsupported (25 factors require 28 runs and raise
`ValueError`). Inspect the actual matrix before budgeting; choose a supported
design or a suitable fractional factorial. Reduced subsets may have better
aliasing properties than the full saturated design, so inspect actual aliases.
See the [current factorial API](https://pydoe.github.io/pydoe/reference/factorial/).
Screen first, optimize later — don't try to learn interactions and find the optimum
in one cheap design.
## Response-surface designs
Two-level designs fit only a flat (linear + interaction) model; they cannot locate an
interior optimum or describe **curvature**. To fit a quadratic and optimize, use a
response-surface methodology (RSM) design over continuous factors:
- **Central composite design (CCD)** (`central_composite`): a 2^k factorial + center
points + axial ("star") points. The axial points add the levels needed to estimate
quadratic terms. With `face="circumscribed"` (default) the axial points sit
*outside* the factorial box (so actual factor levels exceed your stated low/high);
use `face="inscribed"` or `"faced"` to keep everything within the original range.
- **Box-Behnken** (`box_behnken`, needs ≥3 factors): a quadratic design that avoids
the extreme all-low/all-high corners — useful when those corners are unsafe,
expensive, or infeasible. Compare actual run counts and the same center-point
replication: Box-Behnken is not always smaller (for five factors, its 40
non-center runs exceed a half-fraction CCD with 26). The bundled CCD wrapper
uses the full factorial and does not build that half-fraction.
See [NIST response-surface comparisons](https://www.itl.nist.gov/div898/handbook/pri/section3/pri3363.htm).
The wrappers default to a single center run (`center=(0, 1)` for CCD;
`center=1` for Box-Behnken). That supplies no pure-error replication. Pre-plan
independent replicate runs, inspect quadratic-model rank, and budget lack-of-fit
assessment. CCD `center` counts refer to factorial and axial construction blocks;
the wrapper returns a globally shuffled table without block IDs. Preserve or
construct those IDs before using a physically blocked experiment.
Workflow: screen → factorial (find important factors & rough region) → response
surface (model curvature, locate optimum), often moving the experimental region
between steps (path of steepest ascent).
## Space-filling designs
For **computer experiments / simulations** (deterministic or expensive models) where
space coverage may be the main objective, you want even coverage of a
high-dimensional input space:
- **Latin hypercube** (`latin_hypercube`): each factor's range is divided into
n_samples equal bins, sampled once each, arranged to spread points apart
(`criterion="maximin"`). Gives good coverage with relatively few points and is the
common input design for surrogate/emulator modeling. A generic LHS alone is not
a Saltelli/Sobol sensitivity estimator design. Stochastic simulations may still
need replicated random seeds; experimental batches may still need blocking.
## Choosing a design
| Goal | Factors | Design | Script function |
|------|---------|--------|-----------------|
| Screen many factors | 5–20+ | Plackett-Burman / Res III | `plackett_burman` |
| Main effects, some interactions, few runs | 4–8 | Res IV/V fractional | `fractional_factorial` |
| All effects + interactions | 2–5 | Full 2^k factorial | `two_level_factorial` |
| Multi-level categorical | few | Full factorial | `full_factorial` |
| Optimize a response (curvature) | 2–5 | Central composite / Box-Behnken | `central_composite`, `box_behnken` |
| Cover a simulation input space | any | Latin hypercube | `latin_hypercube` |
For physical experiments, **randomize run order within the allowed blocks** to
avoid systematic alignment with drift. DOE wrappers default to a global shuffle;
Latin hypercube defaults to `randomize=False` and already has stochastic sample
placement. Add replicated center points to two-level continuous designs for an
aggregate curvature check; they cannot separate all pure quadratic coefficients.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 323k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 323k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 323k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |