DAAF-Contribution-Community/daaf / .claude/skills/education-data-source-edfacts
education-data-source-edfacts
VerifiedEDFacts — K-12 outcomes: assessment proficiency, ACGR graduation rates, ESSA accountability at school/district level (2009-2020). Within-state trends and subgroup gaps. Complements CCD with outcome data. Cannot compare across states — use NAEP.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 21.1 KB |
| references/accountability-context.md | markdown | 9.6 KB |
| references/assessment-data.md | markdown | 15.5 KB |
| references/data-quality.md | markdown | 17.7 KB |
| references/graduation-rates.md | markdown | 18.4 KB |
| references/subgroup-reporting.md | markdown | 21.1 KB |
| references/variable-definitions.md | markdown | 23.8 KB |
references/graduation-rates.md
View on GitHub# Graduation Rates
Understanding the Adjusted Cohort Graduation Rate (ACGR) methodology, cohort definitions, and proper interpretation.
## Contents
- [What is ACGR?](#what-is-acgr)
- [ACGR Calculation](#acgr-calculation)
- [Cohort Adjustments](#cohort-adjustments)
- [Extended Graduation Rates](#extended-graduation-rates)
- [Diploma Types](#diploma-types)
- [Timing and Reporting](#timing-and-reporting)
- [Cross-State Comparability](#cross-state-comparability)
- [Subgroup Graduation Rates](#subgroup-graduation-rates)
- [Common Pitfalls](#common-pitfalls)
## What is ACGR?
The **Adjusted Cohort Graduation Rate (ACGR)** is the federal standard for calculating high school graduation rates, mandated since 2008 and reported via EDFacts.
### Purpose
- Provide standardized graduation rate calculation
- Enable meaningful comparisons (within states)
- Track student outcomes for accountability
- Required under ESSA for high school accountability
### Historical Context
| Period | Graduation Rate Method |
|--------|------------------------|
| Pre-2008 | States used various methods |
| 2008 | ACGR regulations issued |
| 2010-11 | First ACGR data reported |
| Present | ACGR is federal standard |
## ACGR Calculation
### Basic Formula
```
ACGR = (Graduates with Regular Diploma [+ State-defined alternate diploma, from 2017-18]) / (Adjusted Cohort) × 100
```
### Step-by-Step Calculation
1. **Identify original cohort**: All first-time 9th graders in a given year
2. **Adjust cohort**: Add transfers in, subtract legitimate removals
3. **Count graduates**: Students earning regular diplomas (or, from 2017-18 in States that offer one, a State-defined alternate diploma) within 4 years
4. **Calculate rate**: Graduates ÷ Adjusted Cohort × 100
### What Counts as "On-Time"
| 4-Year ACGR | Definition |
|-------------|------------|
| Cohort entry | Fall of 9th grade year |
| Graduate by | End of 4th year (typically summer) |
| Time allowed | 4 academic years |
### Example Calculation
| Component | Count |
|-----------|-------|
| Original 9th grade cohort (Fall 2019) | 500 |
| Transfers in | +25 |
| Transfers out (documented) | -20 |
| Emigrated | -5 |
| Deceased | -2 |
| **Adjusted cohort** | **498** |
| Graduates with regular diploma (by 2023) | 423 |
| **4-Year ACGR** | **84.9%** |
## Cohort Adjustments
### Legitimate Removals from Cohort
Students can be removed from the cohort only for specific reasons:
| Removal Reason | Documentation Required |
|----------------|------------------------|
| Transfer to another school | Documented enrollment elsewhere |
| Transfer to another country | Documentation of emigration |
| Transfer to private school | Confirmation from receiving school |
| Death | Death certificate |
### Important: What Does NOT Remove from Cohort
| Situation | Impact on Cohort |
|-----------|------------------|
| Dropout | Remains in cohort (non-graduate) |
| GED completion | Remains in cohort (non-graduate in ACGR) |
| Missing/unknown | Remains in cohort (non-graduate) |
| Incarceration | Remains in cohort |
| Homeschool (unverified) | Remains in cohort |
### Transfer Documentation Requirements
States must have documented confirmation that students:
- Enrolled in another public school, OR
- Enrolled in a private school, OR
- Left the country
**If undocumented, student remains in cohort as non-graduate.**
## Extended Graduation Rates
Extended-year ACGRs (e.g., 5- or 6-year) are optional and State-adopted; they are not present in the Portal grad-rates files.
### Rate Types
| Rate | Definition |
|------|------------|
| 4-year ACGR | Graduate within 4 years (required) |
| Extended-year ACGR (e.g., 5-, 6-, or 7-year) | Graduate within additional years; voluntary, and a State may use more than one |
> ED's January 2017 ACGR guidance: "No. The use of an extended-year ACGR is voluntary." A State that adopts one must report it separately from the 4-year ACGR. Extended rates therefore come from State sources, not the Portal files (see `variable-definitions.md` § Extended Rate Variables).
### Why Extended Rates Matter
| Population | Why Important |
|------------|---------------|
| Students with disabilities | May need additional time |
| English learners | Language acquisition takes time |
| Credit-deficient students | May need extra time to recover credits |
| Over-age students | Started high school older |
### Comparing 4-Year to Extended Rates
```python
# Understanding the graduation rate progression
cohort_rates = {
"4-year ACGR": 85.0,
"5-year ACGR": 88.5, # +3.5 pp
"6-year ACGR": 89.2, # +0.7 pp additional
}
# Most late graduates finish in year 5
```
```r
# Understanding the graduation rate progression
cohort_rates <- list(
"4-year ACGR" = 85.0,
"5-year ACGR" = 88.5, # +3.5 pp
"6-year ACGR" = 89.2 # +0.7 pp additional
)
# Most late graduates finish in year 5
```
### Using Extended Rates
| Use Case | Which Rate |
|----------|------------|
| On-time completion | 4-year ACGR |
| Ultimate completion | 5- or 6-year ACGR |
| Students with disabilities | Consider extended rates |
| Full picture | Report all available rates |
## Diploma Types
### Regular Diploma (Counts in ACGR)
Graduates are students receiving a regular high school diploma or (from 2017-18) a State-defined alternate diploma (students with the most significant cognitive disabilities). Before 2017-18, only a **regular high school diploma** counted:
| Counts as Graduate | Does NOT Count |
|--------------------|----------------|
| Standard diploma | GED/equivalency |
| Endorsed diploma | Certificate of completion |
| Honors diploma | Certificate of attendance |
| State-approved regular diploma | Alternative credential |
### Why GED Doesn't Count
| Reason | Explanation |
|--------|-------------|
| Different credential | Not a high school diploma |
| Different pathway | Often pursued after dropout |
| Policy intent | Incentivize diploma completion |
| Federal definition | ACGR specifically requires diploma |
### Alternative Credentials
Students receiving these are counted as **non-graduates** in ACGR:
- General Educational Development (GED)
- High School Equivalency Test (HiSET)
- Certificate of completion
- Diploma based on meeting IEP goals
- Special education certificate
The exception is the State-defined alternate diploma for students with the most significant cognitive disabilities: from 2017-18, in States that offer one, its recipients count as graduates (ED 2017 guidance: "not earlier than ACGR data for the 2017-2018 school year").
### State Variation in Diploma Types
Some states have multiple diploma pathways:
- Standard diploma
- Advanced/honors diploma
- Career/technical diploma
- All count as graduates if state-approved regular diplomas
## Timing and Reporting
### Cohort Year vs. Reporting Year
EDFacts labels ACGR data by the school year in which the cohort graduates: the SY 2023-24 four-year ACGR is for students who first entered grade 9 in SY 2020-21 (class of 2024). Inference from the FS150 SY 2023-24 file specification, whose five-year rate reported for SY 2023-24 covers students who entered grade 9 in SY 2019-20. The Portal follows the same convention: `year` = fall of the graduating school year (year=2019 = SY 2019-20 = class of 2020; 9th-grade entry = `year` − 3).
| Cohort Enters 9th Grade | 4-Year Graduation | EDFacts reporting year | Portal `year` |
|-------------------------|-------------------|------------------------|---------------|
| Fall 2019 | Spring/Summer 2023 | SY 2022-23 | 2022 |
| Fall 2020 | Spring/Summer 2024 | SY 2023-24 | 2023 |
### Release Timing
States submit ACGR data to ED after the cohort's graduation year, and ED/NCES publish it later. The exact submission due dates and public-release lag were not verified (FS150 states no due date); check the EDFacts submission schedule for the year you need. Portal grad-rates years end at 2019.
## Cross-State Comparability
### Better Than Assessment Data
ACGR is **more comparable** across states than proficiency rates because:
| Factor | ACGR | Assessment |
|--------|------|------------|
| Definition | Federally standardized | State-specific |
| Calculation | Same formula everywhere | Different tests |
| Outcome | Diploma yes/no | Test score thresholds |
### Remaining Variations
Despite standardization, some variation exists:
| Source of Variation | Impact |
|---------------------|--------|
| Graduation requirements | States have different course requirements |
| Exit exams | Some states require, some don't |
| Documentation practices | Transfer tracking varies |
| State accountability pressure | May affect practices |
> **Note:** Portal year 2019 (SY 2019-20): "some states may have changed their requirements for a regular high school diploma to account for the impact of the coronavirus pandemic … may have resulted in less comparability in the ACGRs between 2019–20 and other school years" (NCES COE, note 6).
### Interpretation Guidance
| Comparison | Validity |
|------------|----------|
| Within state over time | Valid |
| Across states | Reasonably valid (use caution) |
| Subgroups across states | Reasonably valid (use caution) |
## Subgroup Graduation Rates
### Required Subgroups
ACGR must be reported for these subgroups. In the Portal data, subgroups are represented as **filter columns** with integer codes, not a single "subgroup" column with string codes.
| Subgroup | NCES Code | Portal Column | Portal Filter Value |
|----------|-----------|---------------|---------------------|
| All students | ALL | race (or any filter col) | `99` |
| Economically disadvantaged | ECODIS | econ_disadvantaged | `1` |
| Students with disabilities | CWD | disability | `1` |
| English learners | LEP | lep | `1` |
| Homeless | HOM | homeless | `1` |
| Foster care | FCS | foster_care | `1` |
| Each race/ethnicity | Multiple | race | `1-5, 7` (no `6`; see SKILL.md Race Codes) |
> **Note:** `migrant` and `military_connected` columns are NOT present in grad rate datasets (only in assessment datasets).
### Subgroup Graduation Gaps
Typical patterns in graduation data:
| Comparison | Typical Gap |
|------------|-------------|
| All students vs. SWD | 15-25 percentage points |
| White vs. Black students | 5-15 percentage points |
| White vs. Hispanic students | 5-15 percentage points |
| All students vs. EL | 10-20 percentage points |
### Suppression in Subgroup Data
Small subgroups are heavily suppressed:
- N-size requirements apply
- Smaller schools have more suppression
- Use range/midpoint variables when available
## Common Pitfalls
### Pitfall 1: Confusing Cohort Year and Graduation Year
```python
# Portal grad-rates year = fall of the school year in which the cohort graduates (year=2012 = SY 2012-13, class of 2013; Hawaii Portal 2010/2012 = 80/82 matches NCES ACGR 2010-11/2012-13 = 80/82)
# The cohort entered 9th grade in fall of (year - 3)
cohort_entry_year = year - 3
# Example: year=2019 (SY 2019-20, class of 2020) = Fall 2016 9th-grade cohort
```
```r
# Portal grad-rates year = fall of the school year in which the cohort graduates (year=2012 = SY 2012-13, class of 2013; Hawaii Portal 2010/2012 = 80/82 matches NCES ACGR 2010-11/2012-13 = 80/82)
# The cohort entered 9th grade in fall of (year - 3)
cohort_entry_year <- year - 3
# Example: year=2019 (SY 2019-20, class of 2020) = Fall 2016 9th-grade cohort
```
### Pitfall 2: Treating GED Completers as Graduates
```python
# WRONG: Including GED in graduation count
total_completers = diploma_grads + ged_completers # INCORRECT
# CORRECT: ACGR uses only diploma graduates
acgr_numerator = diploma_grads # Regular diploma (+ State-defined alternate diploma from 2017-18); never GED
```
```r
# WRONG: Including GED in graduation count
total_completers <- diploma_grads + ged_completers # INCORRECT
# CORRECT: ACGR uses only diploma graduates
acgr_numerator <- diploma_grads # Regular diploma (+ State-defined alternate diploma from 2017-18); never GED
```
### Pitfall 3: Ignoring Extended Rates
When analyzing students with disabilities or EL students (extended rates come from States that adopt them, not from the Portal files; the values below are illustrative):
```python
# Limited view
swd_4year = 65.0 # Students with disabilities, 4-year
# Better view
swd_rates = {
"4-year": 65.0,
"5-year": 72.0, # Many complete in year 5
"6-year": 74.5
}
```
```r
# Limited view
swd_4year <- 65.0 # Students with disabilities, 4-year
# Better view
swd_rates <- list(
"4-year" = 65.0,
"5-year" = 72.0, # Many complete in year 5
"6-year" = 74.5
)
```
### Pitfall 4: Comparing Pre- and Post-ACGR Data
| Period | Method | Comparability |
|--------|--------|---------------|
| Before 2010-11 | Various state methods | NOT comparable to ACGR |
| 2010-11 onward | ACGR | Comparable within ACGR era |
### Pitfall 5: Not Accounting for Cohort Adjustments
Schools with high mobility may have:
- Many transfers in/out
- Large cohort adjustments
- Rates affected by documentation quality
## Data Variables in EDFacts
### Graduation Rate Variables
> **Empirically verified** from 2019 grad rate parquet data.
| Variable | Description | Portal Type |
|----------|-------------|-------------|
| `grad_rate_midpt` | Graduation rate, midpoint or exact value | Int64 |
| `grad_rate_low` | Graduation rate, lower bound (null when exact) | Int64 |
| `grad_rate_high` | Graduation rate, upper bound (null when exact) | Int64 |
| `cohort_num` | Number of students in adjusted cohort | Int64 |
> **Note:** The cohort variable is `cohort_num`, NOT `cohort_count`. There is no `grad_count` column. The `grad_rate_midpt` is Int64, not Float64.
### Fetching Graduation Rate Data
```python
# Fetch graduation rate data via mirror system
grad_data = fetch_yearly_from_mirrors(
path_template="edfacts/schools_edfacts_grad_rates_{year}",
years=[2018, 2019],
)
# Filter to California
ca_grads = grad_data.filter(pl.col("fips") == 6)
# Use midpoint for analysis
ca_grads.select([
"ncessch",
"school_name",
"grad_rate_midpt" # Use midpoint
])
```
```r
library(arrow)
library(dplyr)
# fetch_yearly_from_mirrors() is a Python helper; in R, build per-year URLs
# from the mirror root. Mirror failover: see
# `education-data-query/references/fetch-patterns.md` (R pattern)
mirror <- yaml::read_yaml("mirrors.yaml")$mirrors[[1]]
years <- c(2018, 2019)
paths <- sprintf("edfacts/schools_edfacts_grad_rates_%d", years)
urls <- paste0(mirror$root_url, "/", mirror$vintage$hf_revision, "/", paths, ".", mirror$format)
# NOTE: illustrative only — schools_edfacts_grad_rates_* are Polars-written with
# string_view columns, so this plain arrow::read_parquet() fails under R arrow
# ("cannot handle Array of type <utf8_view>"). Real fetch scripts must use the
# view-safe parquet read from `education-data-query/references/fetch-patterns.md`.
grad_data <- bind_rows(lapply(urls, arrow::read_parquet))
# Filter to California
ca_grads <- grad_data |> filter(fips == 6)
# Use midpoint for analysis
ca_grads |> select(ncessch, school_name, grad_rate_midpt)
```
## Analysis Best Practices
### Recommended Approaches
1. **Use appropriate rate for question**
- On-time completion: 4-year ACGR
- Ultimate completion: Extended rates
2. **Account for subgroup differences**
- Report disaggregated rates
- Note suppression
3. **Consider cohort size**
- Small cohorts have more variability
- Weight by cohort size for aggregations
4. **Document limitations**
- Note suppressed data
- Acknowledge cross-state variation
### Example: Comprehensive Graduation Analysis
> **Note:** Grad rate data uses filter columns (race, lep, disability, econ_disadvantaged, etc.) with integer codes, NOT a single "subgroup" column with string codes.
```python
def comprehensive_grad_analysis(df, state_fips, year):
"""Analyze graduation rates with appropriate context."""
state_data = (df
.filter(pl.col("fips") == state_fips)
.filter(pl.col("year") == year)
.filter(pl.col("grad_rate_midpt") >= 0) # Exclude missing codes
)
# Overall rate: race=99 (total), all other filter cols=99 (total)
overall = state_data.filter(
(pl.col("race") == 99) &
(pl.col("lep") == 99) &
(pl.col("disability") == 99) &
(pl.col("econ_disadvantaged") == 99)
)
# Economically disadvantaged students
econ_dis = state_data.filter(pl.col("econ_disadvantaged") == 1)
# Students with disabilities
cwd = state_data.filter(pl.col("disability") == 1)
# English learners
el = state_data.filter(pl.col("lep") == 1)
all_rate = overall["grad_rate_midpt"].mean()
# Note: gaps are calculated per-subgroup filter column
print(f"Overall rate: {all_rate:.1f}")
for label, subgroup_df in [("Econ Dis", econ_dis), ("CWD", cwd), ("EL", el)]:
sub_rate = subgroup_df["grad_rate_midpt"].mean()
print(f"{label}: {sub_rate:.1f} (gap: {sub_rate - all_rate:.1f})")
```
```r
library(dplyr)
# Analyze graduation rates with appropriate context
# NOTE: name the year variable distinctly from the `year` column and unquote it
# with !! — `filter(year == year)` compares the column to itself (always TRUE).
target_year <- 2019
state_data <- df |>
filter(fips == state_fips, year == !!target_year, grad_rate_midpt >= 0)
# Overall rate: race=99 (total), all other filter cols=99 (total)
overall <- state_data |>
filter(race == 99, lep == 99, disability == 99, econ_disadvantaged == 99)
# Subgroup data
econ_dis <- state_data |> filter(econ_disadvantaged == 1)
cwd <- state_data |> filter(disability == 1)
el <- state_data |> filter(lep == 1)
all_rate <- mean(overall$grad_rate_midpt, na.rm = TRUE)
# Note: gaps are calculated per-subgroup filter column
cat(sprintf("Overall rate: %.1f\n", all_rate))
for (info in list(
list("Econ Dis", econ_dis),
list("CWD", cwd),
list("EL", el)
)) {
sub_rate <- mean(info[[2]]$grad_rate_midpt, na.rm = TRUE)
cat(sprintf("%s: %.1f (gap: %.1f)\n", info[[1]], sub_rate, sub_rate - all_rate))
}
```
## Federal Guidance
### Key Regulations
| Regulation | Content |
|------------|---------|
| ESEA §8101(23), (25) (as amended by ESSA) | Statutory four-year and extended-year ACGR definitions; 34 CFR 200.19 is the 2008 regulation |
| ESSA Section 1111 | Graduation rate requirements |
| ED Guidance (2017) | Non-regulatory guidance on ACGR |
### Compliance Requirements
| Requirement | Details |
|-------------|---------|
| Report ACGR | Required for all high schools |
| Report subgroups | If n-size met |
| Include in accountability | High school indicator |
| Extended rates | Optional (State-adopted); if used, reported separately from the 4-year ACGR |
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |