Skip to content

DAAF-Contribution-Community/daaf / .claude/skills/data-scientist

data-scientist

Verified

Data science methodology and method-selection routing for quantitative research. Covers EDA, data validation, descriptive analysis, causal inference (IV, DiD, RD, synthetic control), clustering/PCA/UMAP, supervised ML, geospatial analysis, network analysis, and visualization design. Contains the canonical method-to-library routing tree, routed by execution language — Python: statsmodels (OLS/GLM/time series), pyfixest (FE/IV/DiD), linearmodels (RE/GMM/SUR), svy (complex surveys), scikit-learn (clustering/prediction ML), geopandas (spatial), igraph (network analysis); R: r-stats (OLS/GLM/time series), fixest (FE/IV/DiD), plm (panel RE/between, dynamic GMM), survey-r (complex surveys), tidymodels (ML), sf-terra (spatial), igraph-r (network analysis). Load the routed tool-specific skill before giving tool-specific advice or writing code — library skills encode environment constraints and curated caveats absent from general knowledge.

references/causal-inference.md

View on GitHub
# Causal Inference Methodology

A practitioner's guide to causal reasoning in empirical research. This reference covers
when and why to use causal methods, the assumptions behind each design, and how to
evaluate the credibility of causal claims. Whenever a specific library enters the
discussion — in advice or in code — load the appropriate library skill (e.g.,
`pyfixest`, `statsmodels`): it carries the syntax plus environment constraints and
curated caveats that general knowledge lacks.

> **Package availability:** Where this guide names an estimation package with a
> `pip install X` reference (e.g., pysyncon, synthdid, scpi-pkg, CausalPy, csdid),
> that is a package-identity pointer, not an instruction to run. None of these are
> pre-installed in the DAAF container, and runtime installs are blocked
> (`pip install`/`uv add` are refused both at the command line and inside executed
> scripts — see CLAUDE.md § Runtime Package Installation). Before planning an
> analysis around one, verify availability with `pip show <pkg>` and, if it is
> absent, escalate to the user to add it to the Dockerfile (user additions block
> near the end) and rebuild before use. For the deeper synthetic-control catalog
> see `./causal-synth.md`, which carries the same caveat.

## Contents

- [Acknowledgments](#acknowledgments)
- [The Fundamental Problem of Causal Inference](#the-fundamental-problem-of-causal-inference)
- [The Potential Outcomes Framework](#the-potential-outcomes-framework)
- [Directed Acyclic Graphs (DAGs)](#directed-acyclic-graphs-dags)
- [The Credibility Revolution](#the-credibility-revolution)
- [The Identification Problem](#the-identification-problem)
- [Method Selection Guide](#method-selection-guide)
- [Method-Specific Guidance](#instrumental-variables-key-diagnostics)
- [Modern Difference-in-Differences: A Practitioner's Guide](#modern-difference-in-differences-a-practitioners-guide)
- [Multiple Testing Corrections](#multiple-testing-corrections)
- [Machine Learning for Causal Inference](#machine-learning-for-causal-inference)
- [Causal vs Descriptive: Knowing the Difference](#causal-vs-descriptive-knowing-the-difference)
- [Threats to Validity Checklist](#threats-to-validity-checklist)
- [Partial Identification and Sensitivity Analysis](#partial-identification-and-sensitivity-analysis)
- [References and Further Reading](#references-and-further-reading)

## Acknowledgments

These materials draw extensively from several open-access resources that the authors
have generously made available to the research community:

- **Scott Cunningham's** *Causal Inference: The Mixtape* (freely available at
  mixtape.scunning.com), which provides an exceptionally accessible introduction to
  the potential outcomes framework and quasi-experimental methods
- **Nick Huntington-Klein's** *The Effect* (freely available at theeffectbook.net),
  which offers an intuitive, DAG-centered approach to research design and causality
- **Joshua Angrist and Jorn-Steffen Pischke's** *Mastering 'Metrics* and *Mostly
  Harmless Econometrics*, which established the pedagogical framework of the "furious
  five" quasi-experimental methods and articulated the credibility revolution
- **Pedro Sant'Anna's** comprehensive DiD lecture series (psantanna.com/did-resources),
  which provides the clearest available exposition of why traditional TWFE fails and
  how modern estimators solve the problem
- **Baker, Callaway, Cunningham, Goodman-Bacon, and Sant'Anna's** practitioner's
  guide to DiD, which unifies the modern DiD literature into a coherent framework

Their contributions to accessible econometrics education are immensely valuable and
have shaped how a generation of researchers thinks about causal inference.

## The Fundamental Problem of Causal Inference

The fundamental problem, as articulated by Holland (1986), is deceptively simple: we
can only ever observe one potential outcome per unit. A student either received a
scholarship or did not. A state either expanded Medicaid or did not. We never observe
the same unit under both conditions at the same time.

The counterfactual -- what *would have happened* under the alternative treatment
status -- is fundamentally unobservable. This is not a data limitation that can be
solved by collecting more data. It is a logical impossibility: we cannot simultaneously
observe a treated and untreated version of the same unit.

All of causal inference is about constructing credible approximations to this missing
counterfactual. The methods differ in *how* they approximate it:

- **Randomized experiments** create statistical equivalence between groups by design
- **Regression/matching** conditions on observed characteristics to make groups comparable
- **Instrumental variables** isolates exogenous variation in treatment
- **Regression discontinuity** exploits known policy cutoffs that create quasi-random variation
- **Difference-in-differences** uses parallel temporal trends and differential treatment timing
- **Fixed effects** exploits within-unit variation over time to remove time-invariant confounders
- **Synthetic control** constructs a weighted combination of control units to match the treated unit

Two complementary frameworks formalize this reasoning:

1. **Potential outcomes** (the Rubin causal model): Defines causality in terms of
   Y(1) and Y(0) -- what happens under treatment vs. control
2. **Directed acyclic graphs** (the Pearl framework): Defines causality in terms of
   causal paths, back doors, and d-separation rules

These frameworks are not competing -- they illuminate different aspects of the same
problem. Following Cunningham (2021, Chs. 3-4) and Huntington-Klein (2022; online 2nd
ed., Chs. 5-10), researchers benefit from fluency in both.

## The Potential Outcomes Framework

Following the notation in Cunningham (2021, Ch. 4) and Angrist and Pischke (2009,
Ch. 2):

**Core notation:**
- Y_i(1) = outcome for unit i if treated
- Y_i(0) = outcome for unit i if not treated
- D_i = treatment indicator (1 = treated, 0 = control)
- Y_i = D_i * Y_i(1) + (1 - D_i) * Y_i(0) -- the "switching equation" mapping
  potential outcomes to the single observed outcome

**Treatment effect parameters:**

| Parameter | Definition | What It Captures |
|-----------|-----------|------------------|
| Individual treatment effect | tau_i = Y_i(1) - Y_i(0) | Effect for a specific unit (never observed) |
| ATE | E[Y(1) - Y(0)] | Average effect across the entire population |
| ATT | E[Y(1) - Y(0) \| D=1] | Average effect for those who actually received treatment |
| ATU | E[Y(1) - Y(0) \| D=0] | Average effect for those who did not receive treatment |
| LATE | Effect for compliers (IV) | Average effect for units whose treatment status is changed by the instrument (Imbens and Angrist 1994) |
| CATE | E[Y(1) - Y(0) \| X=x] | Average effect conditional on characteristics -- heterogeneous effects |

**The Simple Difference in Means (SDO) decomposition** (Cunningham 2021) reveals why
naive comparisons fail:

```
E[Y|D=1] - E[Y|D=0] = ATE + Selection Bias + Heterogeneous Treatment Effect Bias
```

Where:
- **ATE** is the quantity of interest
- **Selection bias** = E[Y(0)|D=1] - E[Y(0)|D=0] -- systematic baseline differences
  between treated and control groups (e.g., students who seek scholarships may already
  be higher-performing)
- **Heterogeneous treatment effect bias** = (1 - pi)(ATT - ATU) -- arises when effects
  vary across units and sorting into treatment is non-random

The simple comparison only equals the ATE when both bias terms are zero --
which randomization ensures by design.

**Key assumptions:**
- **Independence**: (Y(1), Y(0)) independent of D -- treatment assignment is unrelated
  to potential outcomes. Randomization guarantees this.
- **SUTVA** (Stable Unit Treatment Value Assumption): (1) each unit receives the same
  treatment dose, (2) no spillovers between units, and (3) no general equilibrium
  effects. SUTVA violations arise in network settings, when implementation quality
  varies, or when scaling changes the effect.

## Directed Acyclic Graphs (DAGs)

DAGs provide a visual language for encoding causal assumptions, following Pearl's
framework as presented in Huntington-Klein (2022; online 2nd ed., Chs. 6-8) and
Cunningham (2021, Ch. 3).

**Core components:**
- **Nodes** represent variables
- **Directed edges** (arrows) represent causal effects
- The absence of an arrow is itself a causal claim (no direct effect)

**Three fundamental structures:**

| Structure | Pattern | Implication |
|-----------|---------|-------------|
| Chain | A -> B -> C | B mediates the effect of A on C |
| Fork | A <- B -> C | B confounds the relationship between A and C |
| Collider | A -> B <- C | Conditioning on B creates a spurious association between A and C |

**Back door paths** are non-causal paths from treatment to outcome that create bias.
The **back door criterion** (Pearl 1995) states: to identify the causal effect of X
on Y, close all back door paths by conditioning on appropriate variables.

**Collider bias** is a critical pitfall: conditioning on a collider *opens* a
previously closed path and creates bias that did not exist before. Common examples
include conditioning on an intermediate outcome (e.g., studying only college graduates
when analyzing the effect of high school quality on earnings) or controlling for a
"bad control" that is itself affected by treatment.

**d-separation rules** formalize when paths are open or closed (Cunningham 2021, Ch. 3):
- A **chain** (A -> B -> C) is *open* unless you condition on B
- A **fork** (A <- B -> C) is *open* unless you condition on B
- A **collider** (A -> B <- C) is *closed* unless you condition on B (or a descendant of B)
- A path is **blocked** (d-separated) if any node along the path blocks information flow
- Two variables are **d-separated** given a conditioning set if *all* paths between
  them are blocked -- this means conditioning removes the statistical association

The practical implication: to identify the causal effect of X on Y, find a
conditioning set that blocks all back-door paths (non-causal associations) while
leaving causal paths open. For complex graphs with more than 5-6 nodes (a DAAF
convention), use dagitty.net (a free web tool) to algorithmically determine valid
adjustment sets.

**Practical DAG construction** (Huntington-Klein 2022; online 2nd ed., Ch. 7):
1. List all variables you believe are relevant to the treatment-outcome relationship
2. Draw arrows based on theory, domain knowledge, and prior literature
3. Simplify by removing variables that are redundant or pure mediators (unless
   mediation is the question)
4. Accept that DAGs encode assumptions -- be explicit about what you are assuming
   and what would change if an assumption is wrong

DAGs complement potential outcomes: the potential outcomes framework defines *what*
we want to estimate (ATE, ATT, etc.); DAGs clarify *what we need to condition on*
(or avoid conditioning on) to identify it.

## The Credibility Revolution

Angrist and Pischke (2010) named a paradigm shift in empirical economics: the move
from structural modeling toward **design-based research** that emphasizes transparent
identification strategies.

**Four framing questions** for empirical research (paraphrasing Angrist and Pischke
2009; not checked against the book):
1. What is the causal relationship of interest?
2. What is the ideal experiment to capture it?
3. What is the identification strategy -- how does your observational data
   approximate that ideal experiment?
4. What is the mode of statistical inference?

The credibility revolution's central insight is that **transparency of assumptions
matters more than mathematical sophistication**. A simple difference-in-differences
with a clearly defended parallel trends assumption is more credible than a complex
structural model with opaque identifying restrictions.

Angrist's 2021 Nobel lecture illustrates design-based evidence graphically, for
example with RD plots and "visual instrumental variable (VIV)" plots (Angrist 2021).

This revolution produced the modern toolkit: regression discontinuity designs
became standard for policy evaluation, DiD methods were formalized and improved,
and instrumental variables were held to stricter relevance standards. The common
thread is asking: "What is the source of identifying variation, and is it credible?"

## The Identification Problem

Selection bias is the principal enemy of causal inference (paraphrasing Angrist and
Pischke 2009; not checked against the book). Units that receive treatment typically
differ systematically from those that do not, and these differences contaminate
naive comparisons.

**Identification** means isolating the meaningful variation in the data -- the
variation that allows a causal interpretation. Huntington-Klein (2022; online 2nd
ed., Ch. 5) frames identification as isolating the variation that answers the
research question.

**Hierarchy of evidence** (for internal validity; a DAAF framing):

| Design | Strength | Why |
|--------|----------|-----|
| Randomized Controlled Trial | Strongest | Randomization eliminates selection bias by design |
| Quasi-experimental (IV, RD, DiD) | Strong | Exploits plausibly exogenous variation from natural experiments |
| Observational with controls | Moderate | Relies on conditional independence assumption -- all confounders observed |
| Naive comparison | Weakest | Subject to both selection bias and omitted variable bias |

This hierarchy is about internal validity (did X cause Y in this sample?), not
external validity (does the finding generalize?). A well-identified local estimate
may not generalize; a poorly identified "representative" estimate may be wrong.

## Method Selection Guide

The table below maps research designs to their assumptions, estimands, and primary
threats. This is the central decision reference for Stage 8 analysis work.

| Design | When to Use | Key Assumptions | What It Estimates | Key Threats | Python Package |
|--------|-------------|-----------------|-------------------|-------------|----------------|
| **Randomized Controlled Trial** | You can randomize treatment | Random assignment, SUTVA, no attrition | ATE | Attrition, non-compliance, external validity | pyfixest, statsmodels |
| **Stratified RCT** | RCT with balance on key characteristics | Random assignment within strata | ATE (with precision gains) | Same as RCT plus stratification errors | pyfixest |
| **Regression with controls** | Selection on observables is plausible | Conditional independence (CIA) | ATE (if CIA holds) | Unobserved confounders, bad controls | pyfixest, statsmodels |
| **Matching / IPW** | Selection on observables, want nonparametric robustness | CIA + common support (overlap) | ATT (matching) or ATE (IPW) | Overlap violations, unobserved confounders | sklearn + statsmodels (see causal-matching.md) |
| **Fixed effects / panel** | Repeated observations, time-invariant confounders | Strict exogeneity (no feedback from Y to future X) | ATE (within-unit) | Time-varying confounders, Nickell bias | pyfixest (`feols`) |
| **Instrumental Variables** | Endogenous treatment, valid instrument exists | Relevance, independence, exclusion restriction, monotonicity | LATE (compliers only) | Weak instruments, exclusion restriction violations | pyfixest (IV formula) |
| **Regression Discontinuity** | Treatment assigned by score crossing a cutoff | Continuity at cutoff, no manipulation | ATE at the cutoff (sharp); LATE for compliers at the cutoff (fuzzy) | Sorting/manipulation, bandwidth sensitivity | rdrobust (installed) |
| **Difference-in-Differences** | Policy change affects some units but not others | Parallel trends, no anticipation | ATT | Parallel trends violations, anticipation | pyfixest (did2s, lpdid) |
| **Synthetic Control** | Few treated units, long pre-treatment period | Weighted controls can match pre-treatment trajectory | ATT for treated unit(s) | No good match, interpolation bias | Manual scipy implementation (see causal-synth.md); a dedicated package (pysyncon, synthdid) is not installed and needs a Dockerfile addition |

*(The package column shows Python implementations; in R, map to `fixest` for FE/IV/DiD
(panel IV included), `plm` for panel RE and dynamic-panel GMM (`pgmm`), and base `stats`
for regression with controls, per the Language Routing table in SKILL.md and the R
library skills' own references. LIML, non-panel GMM-IV, SUR and 3SLS have no installed R
package: escalate to the orchestrator.)*

**Implementation status note:** For Python, the DAAF environment includes `pyfixest`
and `statsmodels` as installed causal inference packages; R ships unconditionally in
the same environment with `fixest` and `plm` installed (load the `fixest`, `plm`, or
`r-stats` skill), covering the same FE/IV/DiD/panel ground. Entries marked with *
require either manual implementation using available packages or installation of
specialized packages:
- **pyfixest** covers: OLS/FE regression, IV (via `~` and `|` formula syntax), panel
  models (`feols` with fixed effects), DiD (`did2s`, `lpdid`, `event_study`), event
  studies, and multiple testing corrections (`rwolf`, `bonferroni`)
- **statsmodels** covers: OLS, GLM, WLS, GLS, and basic time series
- **Matching/IPW**: Propensity scores can be estimated with `scikit-learn`
  (LogisticRegression); matching and weighting require manual implementation
- **RD**: `rdrobust` v1.3.0 is installed (note: `rdplot()` has a bug in this
  version; v2.0.0 fixes it). See `./causal-rd.md` for full implementation
  guidance including bandwidth selection, diagnostics, and visualization workarounds
- **Synthetic control**: No SC package is installed; a manual scipy implementation
  is available as a fallback — see `./causal-synth.md` for manual and package-based
  approaches (pysyncon, synthdid, scpi-pkg and CausalPy each need a Dockerfile
  addition, because runtime installs are blocked)
- **Callaway-Sant'Anna estimator**: Requires installing `csdid`; for staggered DiD
  within the installed environment, use pyfixest's `did2s` or `lpdid`

### Angrist's "Furious Five" Progression

Angrist and Pischke (2015) organize quasi-experimental methods as a pedagogical
progression, in which each method is an increasingly clever way to approximate
random assignment (paraphrasing Angrist and Pischke 2015; not checked against the
book):

1. **Randomized trials** -- the gold standard, but often infeasible or unethical
2. **Regression** -- controls for observables, but cannot handle unobserved confounders
3. **Instrumental variables** -- exploits exogenous variation to overcome endogeneity,
   but requires a valid instrument (strong first stage + exclusion restriction)
4. **Regression discontinuity** -- uses policy cutoffs as natural experiments, but
   estimates are local to the cutoff
5. **Difference-in-differences** -- uses policy changes over time, but requires
   parallel trends to hold

Each method addresses the limitations of the previous one, but introduces its own
assumptions. The progression teaches researchers to match the method to the available
variation in the data, not to pick a method and hope the assumptions hold.

### Huntington-Klein's "Template Matching" Approach

Rather than starting from which method to use, Huntington-Klein (2022; online 2nd
ed., Ch. 12) describes "template" causal diagrams that can be solved easily; the
task is to find the template that matches the research design. Each method
corresponds to a pattern in the causal diagram:

- **RCT**: Treatment is isolated from all back doors by randomization
- **Regression/matching**: Close all back doors by conditioning on observed variables
- **IV**: Route around back-door confounding by exploiting an exogenous instrument
  that affects treatment but has no direct path to the outcome
- **RD**: Exploit a known threshold that creates quasi-random variation near the cutoff
- **DiD**: Exploit parallel temporal trends and differential treatment timing
- **FE**: Exploit within-unit variation over time to close time-invariant back doors

The template approach emphasizes that the research design comes *before* the
statistical method. The DAG determines what you need to condition on; the method
is the tool for implementing that conditioning strategy.

### Instrumental Variables: Key Diagnostics

Beyond the method selection table, IV designs require specific diagnostics because
the assumptions are strong and largely untestable:

**First-stage strength:**
- The first-stage F-statistic tests whether the instrument actually predicts
  treatment. The traditional rule of thumb (Staiger and Stock 1997) is F > 10;
  a stricter rule of thumb, F > 100, is also used, as is formal
  weak-instrument-robust inference.
- With F < 10, IV estimates are biased toward OLS and confidence intervals are
  unreliable. Consider LIML (Limited Information Maximum Likelihood), which is
  less biased than 2SLS with weak instruments. (These are standard results stated
  here without a checked source.)
- Stock and Yogo (2005) provide critical values for testing weak instruments under
  specific bias tolerance thresholds.

**Exclusion restriction:**
- The exclusion restriction (the instrument affects the outcome *only* through
  treatment) **cannot be tested statistically** -- it requires a theoretical defense.
- When multiple instruments are available, the Sargan/Hansen J-test checks
  overidentifying restrictions, but only tests whether instruments agree with
  each other, not whether they are individually valid.

**Monotonicity:**
- The LATE interpretation requires monotonicity: the instrument shifts everyone
  in the same direction (no "defiers" who do the opposite of what the instrument
  encourages). This is a substantive assumption that depends on the context.

**Practical decision rule (DAAF convention):** If the first-stage F < 10, **stop and
reconsider the instrument** before interpreting IV results. Weak-instrument
inference is an active research area but the simplest solution is often finding a
stronger instrument.

### Regression Discontinuity: Design Essentials

**For implementation guidance** (rdrobust API, bandwidth selection, diagnostics,
visualization, gotchas)**:** see `./causal-rd.md`. This section covers the methodology
and identification assumptions.

RD designs exploit known policy cutoffs where treatment is assigned based on a
running variable (score) crossing a threshold. Following Cattaneo, Idrobo, and
Titiunik (2020):

**Sharp vs. fuzzy RD:**
- **Sharp RD**: Treatment is a deterministic function of the score -- everyone
  above the cutoff is treated, everyone below is not. The estimand is the ATE
  at the cutoff.
- **Fuzzy RD**: The cutoff creates a *discontinuity in the probability* of
  treatment, but compliance is imperfect. Essentially an IV design where the
  cutoff is the instrument. The estimand is a LATE for compliers at the cutoff.

**Bandwidth selection:**
- The key implementation choice is the bandwidth -- how much data around the
  cutoff to use. Too narrow = imprecise; too wide = biased.
- Calonico, Cattaneo, and Titiunik (2014) provide data-driven optimal bandwidth
  selection with bias-corrected robust confidence intervals.
- Report results at half and twice the chosen bandwidth (Imbens and Lemieux 2008,
  Sec. 7.5). Cattaneo, Idrobo, and Titiunik (2020, Sec. 5.5) caution that such
  checks are informative only over small ranges around the MSE-optimal bandwidth.
  See `./causal-rd.md` for the full sensitivity grid.

**Polynomial order:**
- Local linear regression (p=1) is the standard choice (Cattaneo, Idrobo, and
  Titiunik 2020, Sec. 4.2). Gelman and Imbens (2019) show that global high-order
  (third or higher) polynomials give noisy estimates, sensitivity to the degree,
  and poor confidence-interval coverage; they recommend local linear or quadratic
  fits.

**Manipulation testing:**
- The McCrary (2008) density test checks for a discontinuity in the running
  variable's density at the cutoff (bunching just above or below), which suggests
  units are sorting around the cutoff. It is most informative when manipulation
  runs in one direction (McCrary 2008).
- **A rejection is a serious red flag.** Applications with an abrupt change in the
  number of observations at the cutoff "will tend to be less credible" (Cattaneo,
  Idrobo, and Titiunik 2020, Sec. 5.2). A rejection does not by itself prove
  identification fails: McCrary (2008) notes a continuous density is "neither
  necessary nor sufficient for identification except under auxiliary
  assumptions." If it rejects, investigate the source, report it prominently, and
  consider a donut RD or an alternative design (see `./causal-rd.md`, Gotcha 15).

**Visual presentation:**
- The standard RD plot: scatter of outcomes against the running variable with
  binned means and fitted curves on each side of the cutoff. The treatment
  effect should be visible as a discontinuity in the fitted curves.

### Matching and Inverse Probability Weighting

**For implementation (code patterns, propensity score estimation, matching
algorithms, IPW weights, doubly robust/AIPW estimation, balance diagnostics,
and inference):** see `./causal-matching.md`. This section covers the methodology
and assumptions; the implementation reference covers how to build it with
scikit-learn, statsmodels, scipy, and polars.

Matching and IPW are selection-on-observables strategies: they assume that
conditional on observed covariates X, treatment assignment is independent of
potential outcomes (the conditional independence assumption, CIA).

**Propensity score methods:**
- The propensity score e(X) = P(D=1|X) summarizes all covariates into a single
  balancing score (Rosenbaum and Rubin 1983). Treated and control units with
  similar propensity scores are comparable on observed covariates.
- Propensity scores can be estimated with logistic regression or flexible ML
  methods (`scikit-learn` LogisticRegression, gradient boosting, etc.)

**Common matching algorithms:**
- **Nearest neighbor**: Match each treated unit to the control with the closest
  propensity score. Simple but can produce poor matches in thin regions.
- **Caliper matching**: Nearest neighbor with a maximum distance threshold --
  unmatched treated units are dropped if no good match exists.
- **Coarsened exact matching (CEM)**: Coarsen continuous covariates into bins
  and match exactly within bins. The analyst fixes the maximum allowed imbalance
  in advance and CEM guarantees it is not exceeded (Iacus, King, and Porro 2012),
  but it may discard many observations.
- **Kernel matching**: Weight all controls by their distance from each treated
  unit using a kernel function. Uses more data but requires bandwidth choice.

**Balance diagnostics (critical step):**
- After matching/weighting, check covariate balance between treated and control.
  |SMD| < 0.1 is a common convention (Austin 2011, citing Normand et al. 2001;
  Austin notes there is no agreed threshold); < 0.05 is a DAAF convention.
- Variance ratios should fall within 0.5-2 (Rubin 2001, as summarized by Stuart
  2010, for the propensity score; applying the band to raw covariates is a DAAF
  convention). See `./causal-matching.md`.
- **If balance is not achieved, the matching has failed** -- do not proceed to
  outcome estimation without adequate balance.

**Common support (overlap):**
- The CIA requires common support: for every value of X, there must be both
  treated and control units. If propensity scores are near 0 or 1 for some
  units, treatment is nearly deterministic and matching is unreliable.
- Trim or restrict analysis to the region of overlap. Report how many
  observations are lost.

**The fundamental limitation:** Matching and IPW cannot address unobserved
confounders. If important variables are unmeasured, the CIA fails. Always
complement matching with sensitivity analysis (Rosenbaum bounds, Oster 2019).

### Causal Mediation: Mechanism Decomposition

**For implementation (statsmodels Mediation class, manual bootstrap, moderated
mediation, multiple mediators, sensitivity analysis, and gotchas):** see
`./causal-mediation.md`. This section covers when mediation is appropriate and
the key identification assumption.

Causal mediation analysis decomposes a total effect of treatment T on outcome Y
into a **natural indirect effect** (NIE, also called ACME -- the effect operating
through a mediator M) and a **natural direct effect** (NDE -- the effect through
all other channels; Pearl 2001). The total effect decomposes as TE = NIE(t) + NDE(1 - t) for
t = 0, 1. The familiar TE = NDE + NIE with a common t holds only with no
treatment-mediator interaction (Imai, Keele, and Tingley 2010; see
`./causal-mediation.md`).

**When mediation is the right question:** Mediation is not an identification
strategy -- it does not establish that T causes Y. Rather, it decomposes an
already-identified causal effect into pathways. The research question must be
about *mechanisms* ("through what channel does T affect Y?"), not about *existence*
of a causal effect.

**The key identification challenge -- sequential ignorability:**
The critical assumption (SI-2) requires that there are no unobserved confounders
of the mediator-outcome relationship, conditional on treatment and covariates.
This assumption is **untestable** and **not guaranteed by randomizing treatment**
-- even in an RCT, treatment randomization does not randomize the mediator.
Sensitivity analysis for violations of SI-2 is mandatory, not optional (a DAAF
stance).

**Distinction from simple "controlling for M":** Adding a mediator to a regression
(Y ~ T + M) does not estimate a well-defined causal mediation effect. The modern
framework (Imai et al. 2010) defines the natural direct and indirect effects as
counterfactual quantities and provides estimation methods (simulation, bootstrap)
that handle nonlinear models correctly.

### Sample Selection Bias: Heckman Correction

**For implementation (manual two-step Probit + OLS with inverse Mills ratio, FIML
via scipy.optimize, bootstrap inference, exclusion restriction diagnostics, and
gotchas):** see `./causal-selection.md`. This section covers when Heckman selection
models are appropriate.

Sample selection bias arises when the outcome variable is observed only for a
non-random subset of the population (Heckman 1979). The classic example is
estimating wage equations when wages are observed only for people who choose to
work -- the working population differs systematically from the non-working
population in ways that correlate with wages.

**When to use Heckman models:**
- The outcome is missing non-randomly (observed only for a selected subset)
- Selection into the observed sample is correlated with the outcome
- You have an **exclusion restriction**: a variable that predicts selection but
  does not directly affect the outcome (analogous to an instrument in IV)

**When NOT to use Heckman models:**
- Selection is on observables only -- matching/IPW is simpler and avoids the
  strong bivariate normality assumption
- No exclusion restriction is available -- identification relies on functional
  form assumptions that are fragile in practice
- The goal is estimating a treatment effect rather than correcting for sample
  selection -- use IV, DiD, or RD instead

**Critical implementation note:** `statsmodels` does **not** have a Heckman module.
The `sm.heckman.Heckman()` API shown in many online tutorials does not exist
(GitHub issue #1921, opened 2014, remains unmerged). Implementation requires
manual Probit + OLS with the inverse Mills ratio, or a hand-coded FIML
log-likelihood via `scipy.optimize`. See `./causal-selection.md` for complete
implementation patterns.

### Synthetic Control: Core Principles

For **implementation guidance** (code patterns, package APIs, manual scipy
implementation, inference procedures, and gotchas), see `./causal-synth.md`.
This section covers methodology only.

Synthetic control methods (Abadie, Diamond, and Hainmueller 2010) are designed for
settings with a small number of treated units (often just one) and many pre-treatment
periods.

**The idea:** Construct a synthetic version of the treated unit as a weighted average
of untreated "donor" units, chosen so the synthetic unit closely matches the treated
unit's pre-treatment trajectory. The treatment effect is the post-treatment gap
between the treated unit and its synthetic counterfactual.

**Key implementation decisions:**
- **Donor pool selection**: Include only units that are plausible comparisons.
  Including irrelevant donors can introduce interpolation bias.
- **Pre-treatment fit**: The synthetic control must closely match the treated
  unit's pre-treatment outcomes. Poor fit = unreliable counterfactual.
- **Predictor variables**: Match on outcome trajectory and key covariates that
  predict the outcome.

**Inference via placebo tests:**
- Standard inference (t-tests, p-values) does not apply with one treated unit.
- Instead, apply the synthetic control method to each untreated unit in turn
  (in-space placebos) and compare the treated unit's gap to the distribution
  of placebo gaps.
- In-time placebos reassign treatment to a pre-treatment date and check that
  no effect appears before the real treatment.

**When to prefer synthetic control over DiD:**
- Few treated units (1-5; rule of thumb) where standard DiD inference is unreliable
- Long pre-treatment periods available for assessing fit quality
- Treatment effect is large enough to be visible against placebo distribution

### Multiple Testing Corrections

When running multiple specifications, testing multiple outcomes, or analyzing
multiple subgroups, the probability of finding at least one spurious significant
result increases rapidly. This is the multiple comparisons problem.

**Correction methods:**
- **Bonferroni**: Divide the significance threshold by the number of tests.
  Conservative but simple. Available in pyfixest: `bonferroni()` (Python); in R,
  use base `stats::p.adjust(p, method = "bonferroni")` — R's `fixest` does not
  export a Bonferroni routine.
- **Holm (step-down Bonferroni)**: Less conservative than Bonferroni while still
  controlling the family-wise error rate. Order p-values and apply increasingly
  relaxed thresholds.
- **Benjamini-Hochberg**: Controls the false discovery rate (FDR) rather than
  the family-wise error rate. More powerful when testing many hypotheses and
  tolerating some false positives.
- **Romano-Wolf**: Accounts for dependence among test statistics using resampling.
  DAAF's preferred choice for multiple testing in applied economics (a DAAF
  opinion, not a sourced ranking). Available in
  pyfixest: `rwolf()`. This is a Python-side asymmetry in the DAAF environment:
  R's `fixest` has no Romano-Wolf routine and the `wildrwolf` package is not
  installed — in R, fall back to `stats::p.adjust` methods (holm, BH) or request
  installation of `wildrwolf`.

**When corrections are appropriate:**
- Multiple outcomes tested for the same treatment effect
- Multiple subgroup analyses beyond what was pre-specified
- Specification searches or robustness checks presented as primary results

**When corrections may not be needed:**
- Pre-specified primary and secondary outcomes (report both, but note which is primary)
- Distinct research questions that happen to use the same data
- Exploratory analyses clearly labeled as hypothesis-generating

### Machine Learning for Causal Inference

**For implementation guidance** (manual DML with installed packages, S/T-learner
code, EconML and DoubleML API patterns, causal forests, CATE diagnostics, and
gotchas)**:** see `./causal-ml.md`. This section covers methodology only.

A growing literature integrates machine learning with causal inference frameworks.
These methods use ML's flexibility for nuisance parameter estimation while
maintaining the statistical properties needed for valid causal inference.

**Double/debiased machine learning (DML):**
Chernozhukov et al. (2018) developed DML for estimating treatment effects in
high-dimensional settings. The key idea: use ML to estimate nuisance functions
(propensity scores, outcome models) while maintaining valid inference for the
causal parameter through cross-fitting and Neyman orthogonality. Appropriate
when the number of potential confounders is large relative to the sample size.

**Causal forests (Wager and Athey 2018):**
An extension of random forests designed to estimate conditional average treatment
effects (CATE) -- heterogeneous treatment effects across subpopulations. Useful
for discovering which subgroups benefit most from treatment, but requires a
valid experimental or quasi-experimental design as the foundation.

**When to use causal ML vs. traditional methods:**
- Use DML when you have many potential confounders and want to avoid functional
  form assumptions in nuisance estimation, but still have a clear identification
  strategy (the ML handles high dimensions, not identification)
- Use causal forests when exploring treatment effect heterogeneity after
  establishing a credible causal effect with traditional methods
- Traditional methods remain preferred when the covariate space is moderate,
  the identification strategy is well-understood, and interpretability is paramount
- Causal ML does NOT solve the fundamental problem of identification -- it
  complements, not replaces, credible research designs

**Python ecosystem:** `EconML` (Microsoft), `DoWhy` (Microsoft), `DoubleML`, and
`CausalML` (Uber) are the main packages. None are installed by default in the
DAAF environment; install as needed for specific projects. Manual DML and
S/T-learner implementations using installed packages (scikit-learn, statsmodels,
pyfixest) are documented in `./causal-ml.md`.

## Modern Difference-in-Differences: A Practitioner's Guide

This section covers the most active area of methodological development in applied
econometrics. The material draws heavily from Sant'Anna's DiD lecture series,
Callaway and Sant'Anna (2021), and the Baker et al. (2026) practitioner's
guide. Understanding this section is important because DiD is the most common
quasi-experimental design in applied research, and the traditional implementation
(two-way fixed effects regression) has been shown to produce misleading results
in common settings.

### Why Traditional TWFE Fails with Staggered Treatment

Two-way fixed effects (TWFE) regression with staggered treatment timing -- where
different units begin treatment at different times -- produces estimates that can
be severely biased when treatment effects are heterogeneous. The core problems:

1. **Already-treated units serve as controls**: TWFE implicitly uses units that were
   treated earlier as controls for units treated later, comparing post-treatment
   outcomes across groups that are all treated
2. **Negative weights**: Some group-time treatment effects receive negative weights
   in the TWFE estimate, meaning the aggregate coefficient can have the wrong sign
   even if all underlying effects are positive
3. **Conflation of heterogeneity with dynamics**: TWFE event study plots can show
   spurious pre-trends or post-treatment dynamics that are artifacts of the weighting
   problem, not real treatment effect patterns

Three key decomposition results formalize these problems:

- **Goodman-Bacon (2021)** decomposed the TWFE estimator into all the implicit 2x2
  DiD comparisons it combines, revealing that it includes "forbidden comparisons"
  that use already-treated units as controls (Roth et al. 2023)
- **de Chaisemartin and D'Haultfoeuille (2020)** showed directly that some
  group-period average treatment effects receive negative weights in TWFE, so the
  aggregate is not a convex combination of causal effects
- **Sun and Abraham (2021)** demonstrated that event-study specifications using TWFE
  inherit the same contamination, so pre-trend tests from TWFE event studies can be
  misleading

The implication, as Sant'Anna's lecture series emphasizes, is clear: **avoid TWFE
entirely for staggered designs**. Use modern estimators that separate identification,
estimation, and aggregation. Roth et al. (2023) state a less absolute version: "Use TWFE only
if you're confident in treatment effect homogeneity."

### The Modern DiD Framework

Following Sant'Anna's pedagogical approach, modern DiD cleanly separates three
steps that traditional TWFE conflates:

**Step 1 -- Identification: What are we estimating?**
The fundamental building block is the **group-time average treatment effect on the
treated**: ATT(g,t) for group g at time t, where group g is defined by the period in
which units first receive treatment. Each ATT(g,t) is a well-defined causal parameter
under parallel trends and no-anticipation assumptions.

**Step 2 -- Estimation: How do we compute it?**
Each ATT(g,t) can be estimated using:
- **Outcome regression**: model the untreated potential outcome and impute the
  counterfactual
- **Inverse probability weighting (IPW)**: weight observations to create balance
  between treated and control groups
- **Doubly robust (DR)**: combine both approaches -- consistent if *either* the outcome
  model or the propensity score model is correctly specified (Sant'Anna and Zhao 2020).
  DR is generally preferred for its robustness properties (Baker et al. 2026).

**Step 3 -- Aggregation: How do we summarize heterogeneous effects?**
The granular ATT(g,t) parameters are aggregated into interpretable summaries:
- **Overall ATT**: single summary measure (weighted average across all g,t)
- **Event-time ATT**: effects by time relative to treatment onset (the event study)
- **Group-specific ATT**: separate effects for each treatment cohort
- **Calendar-time ATT**: effects at each calendar time period

This separation is the core insight of the modern framework. Each step involves
explicit choices (what controls to use, which model specification, what weights for
aggregation), and making those choices transparent prevents the implicit, opaque
weighting that makes TWFE unreliable.

Baker et al. (2026) reinforce this with a "forward engineering" principle:
fix the target parameter first (e.g., ATT), then derive the appropriate estimator --
rather than starting from a familiar TWFE regression and reverse-engineering the
assumptions that would justify it.

### Choosing Among Modern Estimators

| Estimator | Best For | Key Feature | Reference |
|-----------|----------|-------------|-----------|
| **Callaway-Sant'Anna** | General staggered DiD | Group-time ATT with flexible aggregation; doubly robust; the most general-purpose modern estimator (DAAF convention) | Callaway and Sant'Anna (2021) |
| **Sun-Abraham** | Event studies with staggered timing | Saturated interaction-weighted estimator; corrects contamination in event-study coefficients | Sun and Abraham (2021) |
| **Gardner (did2s)** | Simple imputation approach | Two-stage: estimate unit/time FE from untreated observations, impute counterfactuals, then estimate treatment effects | Gardner (2022) |
| **Wooldridge ETWFE** | Regression-based approach | Extended TWFE with proper cohort-time interactions; familiar regression framework | Wooldridge (2021) |
| **Borusyak-Jaravel-Spiess** | Imputation with efficiency | OLS imputation estimator; efficient under homoskedastic, serially uncorrelated errors when its parallel-trends assumption holds (Roth et al. 2023) | Borusyak, Jaravel, and Spiess (2024) |
| **Local Projections DiD** | Flexible dynamics | Local projections plus a "clean control" condition; can accommodate non-absorbing (reversible) treatment | Dube et al. (2023) |

**Practical guidance on choosing:**
1. **Start with Callaway-Sant'Anna** as the default (DAAF convention) -- it is the
   most general, supports conditional parallel trends with covariates, and provides
   the clearest separation of the three steps. It requires `csdid`, which is not
   installed (see the Implementation status note above)
2. **Use did2s (Gardner 2022)** for a simple two-stage imputation approach. It is
   robust to treatment-effect heterogeneity, but using all pre-treatment periods
   requires a stronger parallel-trends assumption than Callaway-Sant'Anna's
   (Sant'Anna lecture series; Roth et al. 2023)
3. **Use Local Projections DiD** when treatment can turn on and off (non-absorbing
   treatment) or when you want minimal parametric assumptions on dynamics
4. **Use Sun-Abraham** when your primary output is an event study and you want to
   correct the contamination in TWFE event-study coefficients directly
5. For all estimators, **report ATT(g,t) disaggregated results** alongside any
   aggregate summary -- aggregation always loses information, and treatment effect
   heterogeneity is substantively interesting

### Parallel Trends

The parallel trends assumption is the linchpin of every DiD design: absent treatment,
treated and control groups would have followed the same outcome trajectory.

**Testing parallel trends:**
- Pre-treatment event study coefficients that are all insignificant are usually
  read as support for the design -- the standard "pre-trends test" (Roth et al.
  2023, Sec. 4.3); testing the leads jointly is a DAAF convention
- Passing this test is **neither necessary nor sufficient** for parallel trends
  (Sant'Anna, DiD lecture series): failing to reject may simply reflect low
  statistical power (Roth 2022)
- As Roth et al. (2023) note, "the absence of a significant pre-trend does not
  necessarily imply that parallel trends holds; in fact, these tests often have
  low power." (quoted from the arXiv version, arXiv:2201.01194v3)

**Functional form sensitivity:**
Roth and Sant'Anna (2023, *Econometrica*) demonstrated a critical subtlety: parallel
trends can hold in levels but fail in logs, or vice versa. The choice of outcome
scale is not innocuous -- it changes whether the identifying assumption is satisfied.
This means researchers should test functional form sensitivity rather than assuming
it away.

**Sensitivity analysis for violations:**
Rambachan and Roth (2023) developed the "Honest DiD" framework, which asks: how
robust are the conclusions to plausible violations of parallel trends? Rather than
assuming parallel trends holds exactly, this approach bounds the treatment effect
under specified degrees of violation -- producing confidence sets that are honest
about the uncertainty introduced by potential parallel trends failures.

**Pre-trends are not pre-tests:**
A non-significant pre-trend does not validate the design. Roth (2022) showed that
standard pre-tests have low power against economically meaningful violations. The
absence of evidence (of a pre-trend) is not evidence of absence (of a parallel
trends violation). Researchers should complement pre-trend tests with:
- Substantive arguments for why parallel trends is plausible
- Sensitivity analysis for robustness to violations
- Covariate balance assessments between treated and control groups

### Continuous Treatment DiD

Callaway, Goodman-Bacon, and Sant'Anna (forthcoming, *AER*) extend the DiD framework
to continuous treatment variables -- settings where units receive different doses
rather than a binary treatment.

Key insights from this extension:
- Treatment-on-treated parameters (effects for units at their observed dose) are
  identifiable under parallel trends
- Comparing effects *across* treatment levels is more challenging and requires
  additional assumptions
- Popular TWFE estimands "admit multiple interpretations, depending on the
  underlying causal building block, all having important limitations as
  meaningful summaries of treatment effects"
- The paper introduces alternative estimators that avoid these drawbacks

## Causal vs Descriptive: Knowing the Difference

Not every research question requires causal inference, and not every analysis that
*feels* causal actually identifies a causal effect. Distinguishing between causal
and descriptive work is a matter of intellectual honesty.

**When causal claims are warranted:**
- Your research design credibly addresses the identification problem
- You can articulate the source of identifying variation
- The assumptions required for a causal interpretation are defensible and transparent
- You have tested (or bounded) the sensitivity of results to assumption violations

**When descriptive analysis is the appropriate endpoint:**
- Prevalence questions: "How many students experience X?"
- Trend questions: "How has Y changed over time?"
- Disparity questions: "Does the distribution of Z differ across groups?"
- Distribution questions: "What does the distribution of W look like?"
- See `./descriptive-analysis.md` for comprehensive descriptive methodology guidance

**The "Schrödinger's causal inference" pitfall:**
Haber et al. (2022) found that what they call "Schrödinger's causal inference"
(studies that deny causal interest yet carry causal implications and
recommendations) is common in observational health research. For example,
concluding that a program "should be expanded" based on a cross-sectional
correlation is making an implicit causal claim. Either commit to a
causal design or be honest that descriptive findings are suggestive, not conclusive.

**Descriptive subgroup analysis is not causal heterogeneity analysis:**
Exploring how an association varies across subgroups (e.g., the gender gap in earnings
is larger in some industries than others) is a descriptive exercise. Estimating
heterogeneous *treatment effects* (e.g., the effect of a training program differs by
gender) requires a causal design that identifies effects within each subgroup.

**Know which parameter your design identifies:**
Following Huntington-Klein's (2022; online 2nd ed., Ch. 10) five rules of thumb:
1. Randomization in a representative sample, with no adjustment, gives the ATE
2. Randomization within a group, isolating that group, gives a conditional ATE
3. Closing back doors and using the remaining variation gives a weighted ATE
   (e.g., variance-weighted)
4. Using an untreated group as the treated group's counterfactual gives the ATT
5. Isolating the variation driven by an exogenous variable gives the LATE

Huntington-Klein adds that these are rules of thumb, and the estimation method
also shapes which average is obtained. Separately (DAAF note): RD identifies an
effect local to the cutoff (Cattaneo, Idrobo, and Titiunik 2020).

For causal vs. correlational language guidance in communicating results, see
`./research-questions.md` > "Causal vs. Correlational Language."

## Threats to Validity Checklist

### Internal Validity (by design type)

| Design | Primary Threats | What to Check |
|--------|----------------|---------------|
| RCT | Attrition, non-compliance, spillovers, experimenter demand effects | Balance tables, attrition analysis, ITT vs. LATE comparison |
| Regression/matching | Unobserved confounders (the fundamental problem) | Sensitivity analysis (Oster 2019), robustness to specification |
| IV | Weak instruments, exclusion restriction violations, defiers | First-stage F-statistic (>10; >100 is a stricter rule of thumb), exclusion restriction arguments, monotonicity defense |
| RD | Manipulation/sorting at cutoff, sensitivity to bandwidth and polynomial | McCrary density test, bandwidth sensitivity, multiple polynomial orders |
| DiD | Parallel trends violations, anticipation effects, compositional changes | Pre-trend tests + sensitivity analysis (Rambachan and Roth 2023), event study plots, composition checks |
| Synthetic control | No good match, interpolation bias, few donor units | Pre-treatment fit quality, donor pool restrictions, placebo tests |

### External Validity

- **LATE and local RD effects are inherently local**: IV identifies effects for
  compliers only (the subpopulation whose treatment is changed by the instrument),
  and RD identifies effects at the cutoff only. These may not generalize to the
  broader population.
- **Site-specific effects**: Results from one state's Medicaid expansion may not
  transfer to another state with different demographics and healthcare infrastructure
- **Time-specific effects**: Results from one time period may not hold in another
  (e.g., labor market effects during expansion vs. recession)
- **Scaling effects**: A program that works in a small pilot may not have the same
  effect at scale (general equilibrium effects, implementation quality variation)
- Generalizability requires **explicit argument**, not assumption. State clearly
  what population and setting the results apply to.

### Measurement and Data Quality

- **Classical measurement error** (random noise in the variable) attenuates
  coefficients toward zero -- a known bias direction
- **Non-classical measurement error** (systematic mismeasurement correlated with
  other variables) biases in unpredictable directions
- **Construct validity**: Does the variable actually measure the concept you intend?
  (e.g., "days absent" as a measure of student engagement)
- **Missing data**: Not a validity threat per se, but differential missingness
  between treatment and control groups can create bias. See
  `./descriptive-analysis.md` for the MCAR/MAR/MNAR framework.
- **SUTVA violations**: Spillover effects (one unit's treatment affects another
  unit's outcome) and general equilibrium effects (treatment at scale changes the
  environment for everyone) can invalidate standard estimators

## Partial Identification and Sensitivity Analysis

When point identification requires assumptions that are too strong to be credible,
partial identification and sensitivity analysis offer more honest alternatives.

**Oster (2019) -- Selection on unobservables:**
Asks: how much selection on unobservables would be needed to explain away the
observed result? Uses the change in coefficient and R-squared when adding controls
to bound the bias from unobserved confounders. A result is robust if the implied
degree of unobservable selection needed to eliminate the effect is implausibly large
relative to the observable selection.

**Cinelli and Hazlett (2020) -- Omitted variable bias via partial R-squared:**
Provides a framework for sensitivity analysis that asks: how strongly would an
omitted confounder need to be associated with both treatment and outcome to explain
away the result? Produces sensitivity contour plots that visually communicate
robustness.

**Rambachan and Roth (2023) -- Honest DiD for parallel trends:**
Specific to difference-in-differences designs. Rather than assuming parallel trends
holds exactly, this framework produces confidence sets that are valid under specified
degrees of violation. The researcher bounds how far post-treatment violations may
exceed the observed pre-trends (e.g., at most M̄ times the largest pre-treatment
violation), and the method reports confidence sets valid under that restriction.
The "breakdown" value of M̄ is the largest such violation under which the
conclusion survives (Roth et al. 2023).

**Rosenbaum bounds for matching estimators** (DAAF summary; Rosenbaum's own work is
not cited or checked here):
For designs based on matching or propensity scores, Rosenbaum bounds quantify how
sensitive the conclusion is to a hidden confounder of a given strength. The "gamma"
parameter represents the factor by which an unobserved confounder could change the
odds of treatment; the analysis reports the gamma at which the conclusion would
be overturned.

**When to use sensitivity analysis:**
- For any observational study making causal claims (not optional -- in DAAF's
  stance, this is a baseline expectation in credible empirical work)
- When assumptions are debatable (e.g., parallel trends, exclusion restriction)
- When the policy stakes are high and conclusions need to be defensible
- When reviewers or stakeholders are likely to question identifying assumptions

See Huntington-Klein (2022; online 2nd ed., Ch. 21, "Partial Identification") for
an accessible overview of sensitivity analysis across designs.

## References and Further Reading

Section and chapter numbers cited in the text follow the versions read for this
file: arXiv:2201.01194v3 for Roth et al. (2023); arXiv:1911.09511 for Cattaneo,
Idrobo, and Titiunik (2020); the 2007 NBER working paper for Imbens and Lemieux
(2008); and the online 2nd edition (theeffectbook.net) for Huntington-Klein
(2022). Published numbering may differ.

### Textbooks (Open Access)

Cunningham, S. (2021). *Causal Inference: The Mixtape*. Yale University Press. https://mixtape.scunning.com/

Huntington-Klein, N. (2022). *The Effect: An Introduction to Research Design and Causality*. Chapman & Hall/CRC. https://theeffectbook.net/

### Textbooks

Angrist, J.D. and Pischke, J.-S. (2009). *Mostly Harmless Econometrics: An Empiricist's Companion*. Princeton University Press.

Angrist, J.D. and Pischke, J.-S. (2015). *Mastering 'Metrics: The Path from Cause to Effect*. Princeton University Press.

### Key Papers -- Foundations

Angrist, J.D. (2021). "Empirical Strategies in Economics: Illuminating the Path from Cause to Effect." Nobel Prize Lecture. https://www.nobelprize.org/prizes/economic-sciences/2021/angrist/lecture/

Angrist, J.D. and Pischke, J.-S. (2010). "The Credibility Revolution in Empirical Economics: How Better Research Design is Taking the Con out of Econometrics." *Journal of Economic Perspectives*, 24(2), 3-30.

Holland, P.W. (1986). "Statistics and Causal Inference." *Journal of the American Statistical Association*, 81(396), 945-960.

Imbens, G.W. and Angrist, J.D. (1994). "Identification and Estimation of Local Average Treatment Effects." *Econometrica*, 62(2), 467-475.

Pearl, J. (1995). "Causal Diagrams for Empirical Research." *Biometrika*, 82(4), 669-688.

### Key Papers -- Difference-in-Differences

Baker, A., Callaway, B., Cunningham, S., Goodman-Bacon, A., and Sant'Anna, P.H.C. (2026). "Difference-in-Differences Designs: A Practitioner's Guide." *Journal of Economic Literature*, 64(2), 498-557. https://doi.org/10.1257/jel.20251650 (preprint: arXiv:2503.13323)

Baker, A., Larcker, D.F., and Wang, C.C.Y. (2022). "How Much Should We Trust Staggered Difference-in-Differences Estimates?" *Journal of Financial Economics*, 144(2), 370-395.

Borusyak, K., Jaravel, X., and Spiess, J. (2024). "Revisiting Event-Study Designs: Robust and Efficient Estimation." *Review of Economic Studies*, 91(6), 3253-3285.

Caetano, C., Callaway, B., Payne, S., and Rodrigues, H.S. (2024). "Difference in Differences with Time-Varying Covariates." arXiv:2202.02903 (v3).

Callaway, B. and Sant'Anna, P.H.C. (2021). "Difference-in-Differences with Multiple Time Periods." *Journal of Econometrics*, 225(2), 200-230. https://doi.org/10.1016/j.jeconom.2020.12.001

Callaway, B., Goodman-Bacon, A., and Sant'Anna, P.H.C. (forthcoming, 2026). "Difference-in-Differences with a Continuous Treatment." *American Economic Review*. arXiv:2107.02637. (Listed as forthcoming at the AER, 2026, on the first author's research page, checked 2026-10-12; no published journal record yet.)

de Chaisemartin, C. and D'Haultfoeuille, X. (2020). "Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects." *American Economic Review*, 110(9), 2964-2996.

Dube, A., Girardi, D., Jordà, Ò., and Taylor, A.M. (2023). "A Local Projections Approach to Difference-in-Differences." NBER Working Paper 31184. https://doi.org/10.3386/w31184

Gardner, J. (2022). "Two-Stage Differences in Differences." arXiv:2207.05943.

Goodman-Bacon, A. (2021). "Difference-in-Differences with Variation in Treatment Timing." *Journal of Econometrics*, 225(2), 254-277.

Roth, J. (2022). "Pretest with Caution: Event-Study Estimates after Testing for Parallel Trends." *American Economic Review: Insights*, 4(3), 305-322.

Roth, J. and Sant'Anna, P.H.C. (2023). "When Is Parallel Trends Sensitive to Functional Form?" *Econometrica*, 91(2), 737-747. https://doi.org/10.3982/ECTA19402

Roth, J., Sant'Anna, P.H.C., Bilinski, A., and Poe, J. (2023). "What's Trending in Difference-in-Differences?" *Journal of Econometrics*, 235(2), 2218-2244. https://doi.org/10.1016/j.jeconom.2023.03.008

Sant'Anna, P.H.C. and Zhao, J. (2020). "Doubly Robust Difference-in-Differences Estimators." *Journal of Econometrics*, 219(1), 101-122. https://doi.org/10.1016/j.jeconom.2020.06.003

Sun, L. and Abraham, S. (2021). "Estimating Dynamic Treatment Effects in Event Studies with Heterogeneous Treatment Effects." *Journal of Econometrics*, 225(2), 175-199.

Wooldridge, J.M. (2021). "Two-Way Fixed Effects, the Two-Way Mundlak Regression, and Difference-in-Differences Estimators." Working Paper. https://doi.org/10.2139/ssrn.3906345

### Key Papers -- Methods

Abadie, A., Diamond, A., and Hainmueller, J. (2010). "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California's Tobacco Control Program." *Journal of the American Statistical Association*, 105(490), 493-505.

Austin, P.C. (2011). "An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies." *Multivariate Behavioral Research*, 46(3), 399-424. https://doi.org/10.1080/00273171.2011.568786

Calonico, S., Cattaneo, M.D., and Titiunik, R. (2014). "Robust Nonparametric Confidence Intervals for Regression-Discontinuity Designs." *Econometrica*, 82(6), 2295-2326.

Cattaneo, M.D., Idrobo, N., and Titiunik, R. (2020). *A Practical Introduction to Regression Discontinuity Designs: Foundations*. Cambridge University Press.

Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., and Robins, J. (2018). "Double/Debiased Machine Learning for Treatment and Structural Parameters." *Econometrics Journal*, 21(1), C1-C68.

Gelman, A. and Imbens, G.W. (2019). "Why High-Order Polynomials Should Not Be Used in Regression Discontinuity Designs." *Journal of Business & Economic Statistics*, 37(3), 447-456.

Heckman, J.J. (1979). "Sample Selection Bias as a Specification Error." *Econometrica*, 47(1), 153-161. https://doi.org/10.2307/1912352

Iacus, S.M., King, G., and Porro, G. (2012). "Causal Inference without Balance Checking: Coarsened Exact Matching." *Political Analysis*, 20(1), 1-24. https://doi.org/10.1093/pan/mpr013

Imai, K., Keele, L., and Tingley, D. (2010). "A General Approach to Causal Mediation Analysis." *Psychological Methods*, 15(4), 309-334. https://doi.org/10.1037/a0020761

Imbens, G.W. and Lemieux, T. (2008). "Regression Discontinuity Designs: A Guide to Practice." *Journal of Econometrics*, 142(2), 615-635. https://doi.org/10.1016/j.jeconom.2007.05.001

McCrary, J. (2008). "Manipulation of the Running Variable in the Regression Discontinuity Design: A Density Test." *Journal of Econometrics*, 142(2), 698-714.

Pearl, J. (2001). "Direct and Indirect Effects." *Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence (UAI)*, 411-420. arXiv:1301.2300. https://arxiv.org/abs/1301.2300

Rosenbaum, P.R. and Rubin, D.B. (1983). "The Central Role of the Propensity Score in Observational Studies for Causal Effects." *Biometrika*, 70(1), 41-55.

Rubin, D.B. (2001). "Using Propensity Scores to Help Design Observational Studies: Application to the Tobacco Litigation." *Health Services and Outcomes Research Methodology*, 2, 169-188. https://doi.org/10.1023/A:1020363010465

Staiger, D. and Stock, J.H. (1997). "Instrumental Variables Regression with Weak Instruments." *Econometrica*, 65(3), 557-586.

Stock, J.H. and Yogo, M. (2005). "Testing for Weak Instruments in Linear IV Regression." In Andrews, D.W.K. and Stock, J.H. (Eds.), *Identification and Inference for Econometric Models*, 80-108. Cambridge University Press.

Stuart, E.A. (2010). "Matching Methods for Causal Inference: A Review and a Look Forward." *Statistical Science*, 25(1), 1-21. https://doi.org/10.1214/09-STS313

Wager, S. and Athey, S. (2018). "Estimation and Inference of Heterogeneous Treatment Effects Using Random Forests." *Journal of the American Statistical Association*, 113(523), 1228-1242.

### Key Papers -- Sensitivity Analysis

Cinelli, C. and Hazlett, C. (2020). "Making Sense of Sensitivity: Extending Omitted Variable Bias." *Journal of the Royal Statistical Society: Series B*, 82(1), 39-67.

Oster, E. (2019). "Unobservable Selection and Coefficient Stability: Theory and Evidence." *Journal of Business & Economic Statistics*, 37(2), 187-204.

Rambachan, A. and Roth, J. (2023). "A More Credible Approach to Parallel Trends." *Review of Economic Studies*, 90(5), 2555-2591.

### Causal Language in Research

Haber, N.A. et al. (2022). "Causal and Associational Language in Observational Health Research: A Systematic Evaluation." *American Journal of Epidemiology*, 191(12), 2084-2097.

### Teaching Resources

Angrist, J.D. "Mastering Econometrics." Marginal Revolution University. https://mru.org/mastering-econometrics-joshua-angrist

LOST (Library of Statistical Techniques). https://lost-stats.github.io/

Sant'Anna, P.H.C. "DiD Resources." Web page, including "A Comprehensive Course on DiD" (14 slide decks). https://psantanna.com/did-resources/ (accessed 2026-10-12)
brainstormingobra/superpowersYou MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation).Stars: 322kUpdated Claude CodeCodexVerifiedInstall
brand-guidelinesanthropics/skillsApplies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
frontend-designanthropics/skillsGuidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
mcp-builderanthropics/skillsGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK).Stars: 198kUpdated Claude CodeCodexVerifiedInstall