Skip to content

DAAF-Contribution-Community/daaf / .claude/skills/education-data-source-meps

education-data-source-meps

Verified

MEPS — Urban Institute modeled school-level poverty (% at 100% FPL) for public schools. MEPS 2.0 (released 2025-12-11) covers school years 2009-10 through 2022-23 and is planned for annual updates; Portal year 2022 returned rows while 2023 was empty on 2026-08-06 (re-probed). Use when FRPL is unreliable due to CEP and for consistent cross-state measurement.

FileKindSize
SKILL.mdskill md17.5 KB
references/comparison-to-frpl.mdmarkdown9.7 KB
references/data-quality.mdmarkdown15.6 KB
references/data-sources.mdmarkdown11.0 KB
references/methodology.mdmarkdown11.4 KB
references/variable-definitions.mdmarkdown13.7 KB

references/variable-definitions.md

View on GitHub
# MEPS Variable Definitions

Comprehensive definitions of all variables in the MEPS dataset, including identifiers, estimates, and metadata.

> **Codebook Authority:** The variable definitions in this document are summaries for convenience.
> The authoritative source for variable names, codes, and definitions is the codebook `.xls` file
> available in the data mirrors. Use `get_codebook_url("meps/codebook_schools_meps")` from `fetch-patterns.md`
> to download the codebook. If this document contradicts the codebook, trust the codebook and
> flag the discrepancy.

> **CRITICAL: Portal Integer Encoding**
>
> This document describes **Education Data Portal** encodings. The Portal returns all ID and categorical columns as **integers**, not strings. Missing values use **native nulls**, not negative coded values.
>
> | Column | Portal Type | Documentation May Say | Actual Portal Value |
> |--------|-------------|----------------------|---------------------|
> | `ncessch` | Int64 | "12-character string" | `10000200277` (integer) |
> | `fips` | Int64 | "FIPS code" | `6` (integer for California) |
> | `leaid` | Int64 | "7-character string" | `100002` (integer) |
> | Missing | null | "-1, -2, -3" | Native null (no coded values) |

## Core MEPS Variables

### School Identifier

| Variable | Portal Type | Description |
|----------|-------------|-------------|
| `ncessch` | **Int64** | NCES school identification number. Unique identifier for each school. Structure: {2-digit state FIPS}{5-digit district}{5-digit school} |
| `ncessch_num` | **Int64** | Numeric duplicate of NCES school ID (same values) |

**Example**: `60000100001` (as integer)
- `06` = California (state FIPS)
- `00001` = District code
- `00001` = School code

> **Note:** Portal returns as integer. Leading zeros are implicit. To reconstruct 12-character string: `f"{ncessch:012d}"`

### Geographic Identifiers (Portal Integer Encoding)

| Variable | Portal Type | Description |
|----------|-------------|-------------|
| `fips` | **Int64** | Federal Information Processing Standard state code (1-56) |
| `leaid` | **Int64** | Local Education Agency (district) ID (7-digit structure, returned as integer) |
| `gleaid` | **Int64** | Geographic Local Education Agency ID |
| `year` | **Int64** | School year (fall semester). Example: 2018 = 2018-19 school year |

> **Note:** All IDs returned as integers. To reconstruct 7-character string for leaid: `f"{leaid:07d}"`

### MEPS Estimates (Portal Field Names)

| Portal Variable | Type | Range | Description |
|-----------------|------|-------|-------------|
| `meps_poverty_pct` | Float64 | 0.0-60.5 | Estimated share of students from households at or below 100% FPL |
| `meps_mod_poverty_pct` | Float64 | 0.0-100.0 | Modified MEPS estimate (reallocated up or down so district aggregates match adjusted SAIPE) |
| `meps_poverty_se` | Float64 | 0.5-3.8 | Standard error of the MEPS estimate |
| `meps_poverty_ptl` | Int64 | 1-100 | National percentile of poverty (enrollment-weighted) |
| `meps_mod_poverty_ptl` | Int64 | 1-100 | Modified percentile (enrollment-weighted) |

> **Field name mapping:** Documentation may reference `meps`, `meps_mod`, `meps_se`. Actual Portal fields are `meps_poverty_pct`, `meps_mod_poverty_pct`, `meps_poverty_se`.

## Understanding MEPS Values

### `meps_poverty_pct`

The primary poverty estimate. Interpretation:
- `meps_poverty_pct = 25.0` means 25% of students estimated to be in poverty (at or below 100% FPL)
- Values range from 0.0 to 60.5 (percentage scale)
- National average is approximately 15-20%

### `meps_mod_poverty_pct`

Modified estimate whose district aggregates match adjusted SAIPE:
- Modified MEPS reallocates modeled poor students so each district matches adjusted SAIPE, preserving relative school differences, both up and down
- Created mainly to address the race and ethnicity bias in the original model (Gutierrez & Blagg 2025)
- Missing where the geographic district has no SAIPE value that year
- Urban suggests it for analyses only of schools within large-enrollment districts
- Range: 0.0-100.0%

### `meps_poverty_se`

Standard error quantifying estimation uncertainty (on the same percentage scale as `meps_poverty_pct`):
- Smaller values = more precise estimates
- Larger values = less certainty about the point estimate
- Use for confidence intervals and statistical comparisons
- Actual observed range: 0.52-3.77

**Calculating confidence intervals:**
```
95% CI = meps_poverty_pct +/- (1.96 x meps_poverty_se)
```

## Poverty Threshold: 100% FPL

### Federal Poverty Level (FPL)

MEPS uses 100% of the Federal Poverty Level as its threshold:

| Family Size | 100% FPL (2024) | 130% FPL | 185% FPL |
|-------------|-----------------|----------|----------|
| 1 | $15,060 | $19,578 | $27,861 |
| 2 | $20,440 | $26,572 | $37,814 |
| 3 | $25,820 | $33,566 | $47,767 |
| 4 | $31,200 | $40,560 | $57,720 |
| 5 | $36,580 | $47,554 | $67,673 |
| 6 | $41,960 | $54,548 | $77,626 |

**Note**: HHS poverty guidelines (FRPL basis). MEPS targets SAIPE/Census poverty thresholds. The guidelines are updated annually by the Department of Health and Human Services; the 100% column above is the 2024 guideline for the 48 contiguous states and DC (ASPE).

### Comparison to Other Thresholds

| Measure | Threshold | Meaning |
|---------|-----------|---------|
| MEPS | 100% FPL | Official poverty line |
| FRPL Free | 130% FPL | Free lunch eligibility |
| FRPL Reduced | 185% FPL | Reduced-price lunch eligibility |
| Near poverty | 200% FPL | Common research threshold |

**Key implication**: MEPS captures **deeper poverty** than FRPL. Schools may have low MEPS but higher FRPL if many families are 100-185% FPL.

## Missing Data Handling

> **CRITICAL: MEPS Uses Native Nulls**
>
> Unlike CCD and other sources, MEPS data in the Portal uses **native null values**, not negative coded values. The codebook lists -1, -2, -3 as theoretical possibilities, but actual Portal data contains nulls.

**Codebook codes (theoretical):**
| Code | Meaning |
|------|---------|
| `-1` | Missing/not reported |
| `-2` | Not applicable |
| `-3` | Suppressed for privacy |

**Actual Portal behavior:**
| Value | Meaning |
|-------|---------|
| `null` | Missing/not available (all missing types) |
| Valid number | Data present |

**Handling missing data (correct approach):**
```python
import polars as pl

# Filter to non-null values
valid_data = df.filter(pl.col("meps_poverty_pct").is_not_null())

# Count missing
null_count = df["meps_poverty_pct"].null_count()
```

```r
library(dplyr)

# Filter to non-null values
valid_data <- df |> filter(!is.na(meps_poverty_pct))

# Count missing
null_count <- sum(is.na(df$meps_poverty_pct))
```

**Do NOT use negative value filtering for MEPS:**
```python
# WRONG - MEPS doesn't use coded values
df.filter(pl.col("meps_poverty_pct") >= 0)  # Unnecessary and misleading
```

```r
library(dplyr)

# WRONG - MEPS doesn't use coded values
df |> filter(meps_poverty_pct >= 0)  # Unnecessary and misleading
```

## Derived Variables

### Creating Categorical Poverty Measures

MEPS is continuous; create categories as needed:

```python
import polars as pl

# Policy-relevant thresholds
df = df.with_columns([
    (pl.col("meps_poverty_pct") >= 30.0).alias("high_poverty"),
    (pl.col("meps_poverty_pct") < 10.0).alias("low_poverty"),
])

# Illustrative poverty bands (analyst-chosen cut points; not Title I
# eligibility -- Urban says MEPS should not be used to allocate resources)
df = df.with_columns(
    pl.when(pl.col("meps_poverty_pct") >= 40.0).then(pl.lit("Very High"))
    .when(pl.col("meps_poverty_pct") >= 25.0).then(pl.lit("High"))
    .when(pl.col("meps_poverty_pct") >= 10.0).then(pl.lit("Moderate"))
    .otherwise(pl.lit("Low"))
    .alias("poverty_level")
)
```

```r
library(dplyr)

# Policy-relevant thresholds
df <- df |> mutate(
  high_poverty = meps_poverty_pct >= 30.0,
  low_poverty = meps_poverty_pct < 10.0
)

# Illustrative poverty bands (analyst-chosen cut points; not Title I
# eligibility -- Urban says MEPS should not be used to allocate resources)
df <- df |> mutate(
  poverty_level = case_when(
    meps_poverty_pct >= 40.0 ~ "Very High",
    meps_poverty_pct >= 25.0 ~ "High",
    meps_poverty_pct >= 10.0 ~ "Moderate",
    .default = "Low"
  )
)
```

### School-Level Poverty Count

Estimate the number of students in poverty (requires joining with CCD enrollment data first):

```python
# After joining MEPS with CCD enrollment on ncessch + year:
df = df.with_columns(
    (pl.col("meps_poverty_pct") / 100.0 * pl.col("enrollment")).round(0).alias("poverty_count")
)
```

```r
library(dplyr)

# After joining MEPS with CCD enrollment on ncessch + year:
df <- df |> mutate(
  poverty_count = round(meps_poverty_pct / 100.0 * enrollment, 0)
)
```

### District Aggregation

Aggregate school MEPS to district level (requires enrollment data from CCD join):

```python
# Enrollment-weighted district average
district_meps = (
    df.filter(pl.col("meps_poverty_pct").is_not_null() & pl.col("enrollment").is_not_null())
    .group_by("leaid")
    .agg(
        (pl.col("meps_poverty_pct") * pl.col("enrollment")).sum()
        / pl.col("enrollment").sum()
    ).rename({"meps_poverty_pct": "meps_weighted_avg"})
)
```

```r
library(dplyr)

# Enrollment-weighted district average
district_meps <- df |>
  filter(!is.na(meps_poverty_pct), !is.na(enrollment)) |>
  group_by(leaid) |>
  summarise(
    meps_weighted_avg = sum(meps_poverty_pct * enrollment, na.rm = TRUE) /
      sum(enrollment, na.rm = TRUE),
    .groups = "drop"
  )
```

## Variable Relationships

### MEPS vs FRPL Conceptual Mapping

| FRPL Concept | MEPS Equivalent | Notes |
|--------------|-----------------|-------|
| % FRPL eligible | `meps_poverty_pct` | Different threshold (185% vs 100% FPL) |
| Free lunch % | No direct equivalent | MEPS doesn't distinguish free vs reduced |
| Reduced lunch % | No direct equivalent | Not separately estimated |
| FRPL count | `meps_poverty_pct / 100 * enrollment` | Approximate only (requires CCD join) |

### MEPS and Other Poverty Measures

| Other Measure | Relationship to MEPS | Correlation |
|---------------|---------------------|-------------|
| SAIPE (district) | Model outcome (adjusted SAIPE); only Modified MEPS matches district aggregates | District correlations with adjusted SAIPE in 2022: 0.80 (free-lunch model), 0.79 (direct-cert model) |
| Census tract poverty | Similar but geographic | Not published (no sourced figure) |
| Title I status | Based on different criteria | Not published (no sourced figure) |
| FRPL (non-CEP schools) | Related but different threshold | Not published (no sourced figure) |
| FRPL (CEP schools) | Not comparable | Not published (no sourced figure) |

## Data Type Specifications (Portal Actual Types)

### Polars Schema (from actual Portal data)

```python
# Polars types - matches actual Portal parquet files
import polars as pl

meps_schema = {
    'year': pl.Int64,
    'fips': pl.Int64,
    'gleaid': pl.Int64,
    'ncessch': pl.Int64,           # Integer, not string!
    'meps_poverty_pct': pl.Float64,
    'meps_poverty_se': pl.Float64,
    'meps_mod_poverty_pct': pl.Float64,
    'meps_poverty_ptl': pl.Int64,
    'meps_mod_poverty_ptl': pl.Int64,
    'ncessch_num': pl.Int64,
    'leaid': pl.Int64,             # Integer, not string!
}
```

```r
library(arrow)

# Arrow schema - matches actual Portal parquet files
meps_schema <- schema(
  year = int64(),
  fips = int64(),
  gleaid = int64(),
  ncessch = int64(),           # Integer, not string!
  meps_poverty_pct = float64(),
  meps_poverty_se = float64(),
  meps_mod_poverty_pct = float64(),
  meps_poverty_ptl = int64(),
  meps_mod_poverty_ptl = int64(),
  ncessch_num = int64(),
  leaid = int64()              # Integer, not string!
)
```

### SQL table definition

```sql
-- SQL table definition (INTEGER for IDs per Portal encoding)
CREATE TABLE meps (
    ncessch BIGINT NOT NULL,       -- 12-digit integer
    year INTEGER NOT NULL,
    fips INTEGER,
    leaid BIGINT,                  -- 7-digit integer, nullable
    gleaid BIGINT,
    meps_poverty_pct DECIMAL(6,4),
    meps_mod_poverty_pct DECIMAL(6,4),
    meps_poverty_se DECIMAL(8,6),
    meps_poverty_ptl INTEGER,
    meps_mod_poverty_ptl INTEGER,
    PRIMARY KEY (ncessch, year)
);
```

### Converting IDs to String Format

If you need traditional string-format IDs for joining with other systems:

```python
import polars as pl

# Convert integer IDs to zero-padded strings
df = df.with_columns([
    pl.col("ncessch").cast(pl.Utf8).str.zfill(12).alias("ncessch_str"),
    pl.col("leaid").cast(pl.Utf8).str.zfill(7).alias("leaid_str"),
])
```

```r
library(dplyr)

# Convert integer IDs to zero-padded strings
df <- df |> mutate(
  ncessch_str = stringr::str_pad(as.character(ncessch), width = 12, pad = "0"),
  leaid_str = stringr::str_pad(as.character(leaid), width = 7, pad = "0")
)
```

## Quick Reference Card

| Portal Variable | What it tells you |
|-----------------|-------------------|
| `meps_poverty_pct` | Estimated share of students in poverty (100% FPL), as percentage |
| `meps_mod_poverty_pct` | Estimate adjusted so district aggregates match adjusted SAIPE |
| `meps_poverty_se` | How confident you can be in the estimate |
| `meps_poverty_ptl` | National percentile (enrollment-weighted, 1-100) |
| `meps_mod_poverty_ptl` | Modified percentile (enrollment-weighted, 1-100) |
| `ncessch` | Unique school identifier for joins (Int64) |
| `leaid` | District ID for aggregation (Int64) |
| `year` | Which school year |
| `fips` | Which state (Int64) |

**Most common usage:**
```python
import polars as pl

# "What's the poverty rate at this school?"
result = df.filter(pl.col("ncessch") == 60000100001).select("meps_poverty_pct")
# Returns value like 25.0 (meaning 25% estimated in poverty)
```

```r
library(dplyr)

# "What's the poverty rate at this school?"
result <- df |> filter(ncessch == 60000100001) |> select(meps_poverty_pct)
# Returns value like 25.0 (meaning 25% estimated in poverty)
```
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 197kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall