Skip to content

DAAF-Contribution-Community/daaf / .claude/skills/education-data-context

education-data-context

Verified

Interpretation guidance for Urban Institute Education Data Portal datasets. The Portal is a curation layer over federal data: lowercase variable names, integer-encoded categoricals, standardized missing codes (-1 missing, -2 not applicable, -3 suppressed). Covers year definitions, grade encoding (grade=-1 is Pre-K, not missing), suppression rates, ODC-By licensing, and cross-source join identifiers. Load before analyzing any Portal data. Routes to source-specific deep-dive skills for individual datasets.

references/edfacts-context.md

View on GitHub
# EDFacts Context

EDFacts provides state-reported data on assessment results, graduation rates, and other accountability indicators for K-12 public schools. In this system, EDFacts data is accessed through the Urban Institute Education Data Portal, which converts original NCES string codes to integer encodings and standardizes variable names to lowercase.

## Source Overview

| Attribute | Value |
|-----------|-------|
| Collector | U.S. Department of Education |
| Respondents | State Education Agencies |
| Mandate | ESEA/ESSA accountability requirements |
| Coverage | Public schools and districts |
| Primary Use | Assessment proficiency, graduation rates |
| Available Years | Portal: assessments 2009–2018 and 2020 (no 2019); graduation rates 2010–2019 |

## Critical Limitation: State Assessments Not Comparable

**THE most important caveat for EDFacts:**

### Why You Cannot Compare Across States

| Factor | How It Varies |
|--------|---------------|
| Assessment content | Each state writes own tests |
| Proficiency cut scores | Set by each state |
| Grade-level expectations | State standards differ |
| Test difficulty | Not calibrated across states |
| Scaling method | Different psychometric approaches |

### What This Means

A student who is "proficient" in Mississippi is NOT equivalent to a student who is "proficient" in Massachusetts:
- Cut scores differ dramatically
- Content coverage differs
- Same-ability student might pass in one state, fail in another

### NAEP Is the Exception

The National Assessment of Educational Progress (NAEP) IS comparable across states:
- Same test nationally
- Same cut scores
- Not in EDFacts (separate data source)

## Proficiency Data

### How Proficiency Is Reported

EDFacts reports proficiency as percentages meeting state standards:

| Variable Type | Example |
|---------------|---------|
| Proficiency rate | 45% proficient or above |
| Ranges | 35-50% (when suppressed) |
| Midpoint estimate | 42.5% (calculated from range) |

### Using Range Data

When counts are small, exact percentages are suppressed. Instead, ranges are provided:

```python
# EDFacts proficiency range handling
# Look for variables ending in _midpt for single-value estimates

df.select([
    "proficiency_low",   # Lower bound
    "proficiency_high",  # Upper bound
    "proficiency_midpt"  # Midpoint for analysis
])
```

```r
# EDFacts proficiency range handling
# Look for variables ending in _midpt for single-value estimates

df |> select(proficiency_low, proficiency_high, proficiency_midpt)
```

**Use `_midpt` variables for single-value analysis**, but note uncertainty.

### Proficiency Levels

Most states report:
- Below Basic / Not Meeting
- Basic / Partially Meeting
- Proficient / Meeting
- Advanced / Exceeding

But labels and cut scores vary by state.

## Missing Assessment Data

### 2019-20 Testing Waivers

**Critical**: ED waived the 2019-20 assessment requirement for all 50 States, DC, Puerto Rico and the BIE (85 FR 29935, May 19, 2020):
- The Portal has no assessment file for Portal year 2019 (SY 2019-20)
- Cannot calculate SY 2019-20 proficiency from Portal data (Portal year 2020 = SY 2020-21)

### Other Missing Data

| Reason | Impact |
|--------|--------|
| State testing changes | New test = break in series |
| Grade span changes | Tested grades may change |
| Opt-out movements | Some states had high opt-out rates |
| Natural disasters | Hurricane years may have missing data |

### Check Before Analysis

```python
# Check assessment data availability
def check_assessment_years(df, state_fips):
    state_data = df.filter(pl.col("fips") == state_fips)
    
    available_years = state_data["year"].unique().sort()
    print(f"Available years for state {state_fips}: {available_years}")
    
    # Check for gaps
    all_years = range(min(available_years), max(available_years) + 1)
    missing = set(all_years) - set(available_years)
    if missing:
        print(f"Missing years: {missing}")
```

```r
# Check assessment data availability
state_data <- df |> filter(fips == state_fips)

available_years <- sort(unique(state_data$year))
cat(sprintf("Available years for state %s: %s\n", state_fips, paste(available_years, collapse = ", ")))

# Check for gaps
all_years <- seq(min(available_years), max(available_years))
missing <- setdiff(all_years, available_years)
if (length(missing) > 0) {
  cat(sprintf("Missing years: %s\n", paste(missing, collapse = ", ")))
}
```

## Graduation Rates

### Adjusted Cohort Graduation Rate (ACGR)

EDFacts reports ACGR, the federal accountability measure:

| Aspect | Definition |
|--------|------------|
| Cohort | First-time 9th graders |
| Adjustments | Transfer in/out, emigration, death |
| Denominator | Adjusted cohort (original - out + in) |
| Numerator | Graduate with a regular diploma (from 2017-18, or a State-defined alternate diploma; see Diploma type below) in 4 years |

### Extended Graduation Rates

| Rate | Definition |
|------|------------|
| 4-year | Within 4 years of 9th grade |

Extended-year ACGRs (e.g., 5- or 6-year) are optional and State-adopted; they are not present in the Portal grad-rates files.

### Graduation Rate Caveats

| Issue | Impact |
|-------|--------|
| Diploma type | Regular diploma; from 2017-18 also a State-defined alternate diploma for students with the most significant cognitive disabilities; other alternative credentials excluded |
| GED completers | Counted as non-graduates in ACGR |
| Late graduates | Not counted in 4-year rate |
| Special education | May take longer; State extended-year rates (not in the Portal) may be more appropriate |
| Transfer tracking | States may lose track of transfers |

### Cohort Timing

For `year=2019` graduation data (Portal grad-rates `year` = fall of the school year in which the cohort graduates):
- School year 2019-20, class of 2020
- Cohort entered 9th grade in fall 2016 (`year` − 3)
- (Verified: Hawaii Portal `year` 2010/2012 = 80/82 matches NCES ACGR 2010-11/2012-13 = 80/82)

## Suppression

### Heavy Suppression for Subgroups

EDFacts has significant suppression:

| Subgroup | Suppression Level |
|----------|-------------------|
| All students | Low |
| Major racial groups | Moderate |
| Small racial groups | High |
| Students with disabilities | High |
| English learners | Moderate to high |
| Economically disadvantaged | Moderate |

### Suppression Impact Analysis

```python
# Check suppression before disaggregating
def check_subgroup_suppression(df, variable, subgroup_col):
    return df.group_by(subgroup_col).agg([
        (pl.col(variable) == -3).sum().alias("suppressed"),
        pl.col(variable).count().alias("total")
    ]).with_columns(
        (pl.col("suppressed") / pl.col("total") * 100).alias("pct_suppressed")
    ).sort("pct_suppressed", descending=True)
```

```r
# Check suppression before disaggregating
df |>
  group_by(.data[[subgroup_col]]) |>
  summarise(
    suppressed = sum(.data[[variable]] == -3),
    total = n()
  ) |>
  mutate(pct_suppressed = suppressed / total * 100) |>
  arrange(desc(pct_suppressed))
```

### Why So Much Suppression?

- Small schools have small subgroups
- (Illustrative/unsourced — verify) 10-student minimum for many indicators
- Complementary suppression to prevent calculation
- Rural areas particularly affected

## Known Data Issues

### State-Specific Problems

| State | Year | Issue |
|-------|------|-------|
| Virginia | 2016-17 | Grade 5-8 math proficiency shares inaccurate |
| Various | 2019-20 | COVID testing waivers - no data |

### Testing Irregularities

Some states have had testing irregularities in specific years:
- Check state education agency announcements
- Unusual year-over-year changes may indicate problems
- Exclude affected years from analysis

## Valid vs. Invalid Comparisons

### Valid Comparisons

| Comparison Type | Validity |
|-----------------|----------|
| Same state over time | Valid (with same test) |
| Districts within state | Valid |
| Schools within district | Valid |
| Subgroups within school | Valid (watch suppression) |

### Invalid Comparisons

| Comparison Type | Why Invalid |
|-----------------|-------------|
| State vs. state proficiency | Different tests and cut scores |
| National ranking by proficiency | Meaningless |
| State "improvement" via proficiency | May reflect test/cut score changes |

## Time Series Issues

### Assessment System Changes

States periodically change assessments:
- New standards (Common Core transition; timing illustrative/unsourced — verify)
- New test vendors
- New cut scores

**These create breaks in time series.**

### Identifying Assessment Changes

```python
# Look for suspicious jumps in state data
def identify_assessment_changes(df, state_fips, variable):
    state_data = df.filter(pl.col("fips") == state_fips).sort("year")
    
    # Calculate year-over-year change
    state_data = state_data.with_columns(
        (pl.col(variable) - pl.col(variable).shift(1)).alias("yoy_change")
    )
    
    # Flag large changes (>10 percentage points)
    large_changes = state_data.filter(pl.col("yoy_change").abs() > 10)
    
    return large_changes
```

```r
# Look for suspicious jumps in state data
state_data <- df |>
  filter(fips == state_fips) |>
  arrange(year) |>
  mutate(yoy_change = .data[[variable]] - lag(.data[[variable]]))

# Flag large changes (>10 percentage points)
large_changes <- state_data |> filter(abs(yoy_change) > 10)
```

### Common Transition Years

| Event | Years Affected |
|-------|----------------|
| Common Core standards | (Illustrative/unsourced — verify) 2014-2016 |
| PARCC/SBAC implementation | (Illustrative/unsourced — verify) 2015-2017 |
| ESSA transition | 2017-2018 |
| COVID waivers | SY 2019-20 (Portal assessment year 2019, absent) |

## Using Scale Scores

### When Available

Some EDFacts data includes scale score information:
- More comparable over time within state
- Allows for growth measurement
- Not all variables have scale scores

### Scale Score Advantages

- Not affected by cut score changes (same scale)
- Can measure growth, not just proficiency
- More granular than proficiency categories

## Recommended Practices

### For Any Analysis

1. **Never compare proficiency across states**
2. **Check for assessment system changes in your time period**
3. **Document suppression rates**
4. **Note COVID-19 impact on 2019-20 data**
5. **Use within-state comparisons only**

### For Proficiency Analysis

1. Use within-state trends only
2. Check for test changes in period
3. Use `_midpt` for suppressed ranges
4. Note what "proficient" means in that state

### For Graduation Rate Analysis

1. Use ACGR for federal comparability
2. Extended-year rates are optional and State-adopted, and not in the Portal; source them from the State if needed
3. Remember cohort timing (Portal `year` = fall of the graduating school year; 9th-grade entry = `year` − 3)
4. Account for transfer tracking limitations

### For Subgroup Analysis

1. Check suppression rates first
2. Smaller groups = more suppression
3. May need to aggregate to district/state level
4. Document groups excluded due to suppression

### For Time Series

1. Identify assessment system changes
2. Create separate pre/post periods if needed
3. Exclude COVID-affected years or note clearly
4. Verify indicator definitions haven't changed

## Alternatives for Cross-State Comparison

If you need to compare across states, consider:

| Source | Use |
|--------|-----|
| NAEP | Nationally comparable assessment (sample) |
| SAT/ACT | Participation varies; selection bias |
| AP exam scores | For AP participants only |
| ACGR | Graduation rates more comparable than proficiency |

## Quick Reference Card

| Task | Guidance |
|------|----------|
| Cross-state proficiency | DO NOT COMPARE - not valid |
| Within-state trends | Valid if same test period |
| Graduation rates | ACGR is more comparable |
| Subgroup analysis | Check suppression first |
| 2019-20 data | Mostly missing (COVID waivers) |
| Time series | Check for assessment changes |
| Suppressed values | Use `_midpt` variables |
| National ranking | Invalid for state assessments |
reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall