DAAF-Contribution-Community/daaf / .claude/skills/education-data-source-crdc
education-data-source-crdc
VerifiedCRDC — OCR civil rights collection for U.S. public schools; Portal families have topic-specific coverage, with year labels from 2011 through 2023 (Portal year = fall of the school year, so 2020 = 2020-21). Discipline, course access, harassment, restraint/seclusion by race/sex/disability/EL. Use for civil rights and equity analysis. OCR documents every collection since 2011-12 as a universe collection; 2020-21 (Portal year 2020) is COVID-impacted.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 26.5 KB |
| references/civil-rights-context.md | markdown | 15.6 KB |
| references/collection-methodology.md | markdown | 16.4 KB |
| references/data-elements.md | markdown | 24.2 KB |
| references/data-quality.md | markdown | 22.8 KB |
| references/historical-changes.md | markdown | 16.7 KB |
| references/variable-definitions.md | markdown | 22.7 KB |
references/historical-changes.md
View on GitHub# CRDC Historical Changes
This reference documents how the Civil Rights Data Collection has evolved across collection years, including changes to coverage, variables, and definitions.
## Contents
- [Timeline Overview](#timeline-overview)
- [Pre-Modern CRDC (1968-2008)](#pre-modern-crdc-1968-2008)
- [Modern CRDC Era (2009-Present)](#modern-crdc-era-2009-present)
- [Year-by-Year Changes](#year-by-year-changes)
- [Variable Introduction Timeline](#variable-introduction-timeline)
- [Definition Changes](#definition-changes)
- [Coverage Evolution](#coverage-evolution)
- [Implications for Analysis](#implications-for-analysis)
---
## Timeline Overview
```
1968 First collection (Elementary and Secondary School Civil Rights Report)
↓
1970s Periodic collections, varying scope (1976 was a universe collection)
↓
1980s Continued periodic collection
↓
1990s Collection continued
↓
2000 Universe collection (with 1976, the only pre-2011 universe collections)
↓
2004, 2006 Sample collections (renamed CRDC from 2004)
↓
2009-10 Modern CRDC redesign begins (sample)
↓
2011-12 First modern CRDC; universe (all public schools and districts)
↓
2013-14 Universe collection (16,758 districts; 95,507 schools); chronic absenteeism first collected
↓
2015-16 Universe collection; chronic absenteeism collected again
↓
2017-18 Full universe, expanded variables; chronic absenteeism taken from EDFacts
↓
2020-21 COVID-impacted collection (Portal year 2020)
↓
2021-22 Post-pandemic collection (Portal year 2021)
↓
2023-24 Universe collection; public-use file released August 31, 2026
↓
2025-26 Next collection (OMB-approved July 20, 2026)
```
---
## Pre-Modern CRDC (1968-2008)
### Original Purpose
The collection began in 1968; before the 2004 CRDC it was administered as the **Elementary and Secondary School Civil Rights Report**:
- Created to monitor school desegregation
- Focused on race/ethnicity data
- Supported enforcement of Civil Rights Act of 1964
### Key Characteristics
| Attribute | Description |
|-----------|-------------|
| Name | Elementary and Secondary School Civil Rights Report (before 2004) |
| Frequency | Irregular (not consistently biennial) |
| Coverage | Sampled districts, except the universe collections of 1976 and 2000 |
| Focus | Desegregation, race data |
| Technology | Paper-based, later electronic |
### Limitations of Pre-Modern Data
- **Not comparable** to modern CRDC
- Different collection instruments
- Different variable definitions
- Different sampling approaches
- Limited electronic availability
---
## Modern CRDC Era (2009-Present)
### The 2009-10 Redesign
Major transformation of the collection:
- New collection system developed
- Expanded data elements
- Focus on educational opportunity
- NCES partnership for quality improvement
### Key Changes from Pre-Modern
| Aspect | Pre-Modern | Modern |
|--------|------------|--------|
| Focus | Desegregation | Educational opportunity |
| Collection | Paper/manual | Web-based |
| Variables | Limited | Extensive |
| Frequency | Irregular | Often described as biennial; no odd/even-year rule (2020-21 and 2021-22 consecutive) |
| Coverage | Sample (except 1976 and 2000) | Universe (every collection from 2011-12 on, per OCR) |
---
## Year-by-Year Changes
### 2011-12 Collection
**First modern CRDC**
| Aspect | Detail |
|--------|--------|
| Coverage | Universe: all public schools and districts (response: 98.4% of districts, 99.2% of schools, 99.6% of students) |
| New elements | Restraint/seclusion, preschool discipline |
| Form structure | Redesigned school and LEA forms |
| Submission | Web-based system |
**Key data elements introduced**:
- Restraint and seclusion
- Preschool suspensions and expulsions
- Harassment/bullying categories
- Advanced course access
- Staff qualifications
### 2013-14 Collection
**Universe collection**
| Aspect | Detail |
|--------|--------|
| Coverage | 16,758 districts and 95,507 schools |
| Changes | Minor refinements to definitions |
| New elements | Some optional elements added |
The official 2013-14 first-look document explicitly identifies this as a **universe collection**. Earlier wording that characterized it as anything less than universe coverage was incorrect. Definitions were clarified based on earlier collection experience, and optional elements were added. Source: 2013-14 CRDC First Look, https://civilrightsdata.ed.gov/assets/downloads/2013-14-first-look.pdf (fetched 2026-10-15).
The First Look reports 2013-14 data on, and lists among its "NEW Data topics", chronic student absenteeism, teacher absenteeism (teachers absent more than 10 school days), and sworn law enforcement/school resource officers. Several items were optional in 2013-14 and required of every school from 2015-16, including OSS instances and school days missed due to OSS, sexual-orientation and religion harassment allegations, and security guard and sworn law enforcement/SRO counts.
### 2015-16 Collection
**Universe collection**
| Aspect | Detail |
|--------|--------|
| Coverage | Universe (96,440 schools, final target) |
| New elements | Items optional in 2013-14 became required; preschool expansion |
| Significance | Last collection with chronic absenteeism on the CRDC form (first collected 2013-14) |
**Major additions**:
- **Chronic absenteeism** - Collected again (first collected 2013-14)
- **OSS instances and school days missed** - Required from every school (optional in 2013-14)
- Expanded preschool data
- Sexual harassment of students
**What this means**:
- Comprehensive school-level coverage
- Second chronic-absenteeism year; from 2017-18 the measure comes from EDFacts
- Longitudinal use still requires definition and availability checks
### 2017-18 Collection
**Full universe, major expansion**
| Aspect | Detail |
|--------|--------|
| Coverage | Full universe (~97,600 schools; 17,637 LEAs final target) |
| New elements | Computer science, expanded staffing |
| Puerto Rico | First inclusion |
| Chronic absenteeism | Dropped from the CRDC form; OCR takes it from EDFacts (state-reported). Treat the 2015 to 2017 change as a source change |
**Major additions**:
- **Computer science offerings and enrollment** (new optional items)
- Internet and device access items (new optional items)
- Puerto Rico schools (first time)
**Expanded elements**:
- More detailed staffing variables
- Additional course offerings
- Refined harassment categories
### 2020-21 Collection
**COVID-impacted year**
| Aspect | Detail |
|--------|--------|
| Coverage | Universe (~97,500 schools) |
| Portal year | 2020 |
| Special context | COVID-19 pandemic |
| New elements | COVID-specific variables |
**COVID-specific additions**:
- Remote instruction amount
- Student remote learning percentage
- Internet and device access items carried over (new optional in 2017-18; not a COVID addition)
**Important notes**:
- Data reflects pandemic disruption
- Many schools operated remotely
- Discipline data artificially low
- **NOT COMPARABLE** to other years for trends
**Not new in 2020-21**: suspension instances and school days missed due to OSS were optional in 2013-14 and required from 2015-16 (Portal: 2015+).
### 2021-22 Collection
**Post-pandemic baseline**
| Aspect | Detail |
|--------|--------|
| Coverage | Universe (~98,000 schools) |
| Portal year | 2021 |
| Context | Return to in-person learning |
| Changes | COVID variables retained; referral and arrest definitions revised |
**Significance**:
- First "normal" year post-pandemic
- Better baseline for future comparisons
- COVID variables included for transition
- Referrals to law enforcement and school-related arrests are not directly comparable to earlier years (2021-22 definition revisions)
### 2023-24 Collection
**Most recent released collection**
| Aspect | Detail |
|--------|--------|
| Coverage | Universe (17,591 LEAs; 97,094 schools) |
| Release | Public-use file and user's manual released August 31, 2026 |
| Portal year | 2023 (exposed for some topics, e.g., offerings, verified 2026-10-15; check per family) |
| Next collection | 2025-26 (OMB-approved July 20, 2026) |
---
## Variable Introduction Timeline
### Discipline Variables
| Variable | First Year |
|----------|------------|
| In-school suspension | 2011 |
| Out-of-school suspension | 2011 |
| Expulsion | 2011 |
| Referral to law enforcement | 2011 |
| School-related arrest | 2011 |
| Corporal punishment | 2011 |
| Preschool suspension | 2011 |
| Suspension instances | 2013 (optional; required 2015-16; Portal: 2015+) |
| School days missed | 2013 (optional; required 2015-16; Portal: 2015+) |
### Restraint and Seclusion
| Variable | First Year |
|----------|------------|
| Mechanical restraint | 2011 |
| Physical restraint | 2011 |
| Seclusion | 2011 |
| Instance counts | 2011 |
### Harassment
| Variable | First Year |
|----------|------------|
| Race/color/national origin harassment | 2011 |
| Sex-based harassment | 2011 |
| Disability harassment | 2011 |
| Sexual orientation harassment | 2015 (optional in 2013-14) |
| Religion harassment | 2015 (optional in 2013-14) |
### Chronic Absenteeism
| Variable | First Year |
|----------|------------|
| Chronic absenteeism (15+ days) | 2013 (CRDC form 2013-14 and 2015-16; from EDFacts from 2017-18) |
### Course Access
| Variable | First Year |
|----------|------------|
| AP course offerings | 2011 |
| AP enrollment by subject | 2011 |
| IB enrollment | 2011 |
| Gifted/talented | 2011 |
| Algebra I/II, Geometry | 2011 |
| Chemistry, Physics | 2011 |
| Computer science | 2017 |
### Staffing
| Variable | First Year |
|----------|------------|
| Teacher FTE | 2011 |
| Teacher experience | 2011 |
| Teacher certification | 2011 |
| Counselor FTE | 2011 |
| Nurse FTE | 2011 |
| School resource officers / sworn law enforcement | 2013 (FTEs required 2015-16) |
| Teacher absenteeism | 2013 |
---
## Definition Changes
### Suspension Definition Evolution
| Year | Definition |
|------|------------|
| 2011-12 onward | Separate counts of students receiving one and more than one OSS (reported in OCR's 2011-12 Data Snapshot) |
### Harassment Categories
| Year | Categories |
|------|------------|
| 2011-14 | Race, sex, disability (3 categories) |
| 2015+ | Added sexual orientation, religion (5 categories; optional in 2013-14) |
### Chronic Absenteeism
| Year | Definition |
|------|------------|
| Pre-2013 | Not collected |
| 2013-14, 2015-16 | Collected on the CRDC form: students missing 15 or more school days |
| 2017-18 onward | Taken from EDFacts (state-reported) rather than the CRDC form; treat the 2015 to 2017 change as a source change |
| 2020-21 | From EDFacts; reflects pandemic attendance disruption; compare with caution |
### Restraint Definition Clarification
| Year | Change |
|------|--------|
| 2011-14 | Initial definitions |
| 2015+ | Clarified exclusion of physical escort |
| 2017+ | Additional guidance on timeout vs. seclusion |
---
## Coverage Evolution
### Collection Coverage Timeline
```
1968-2010 Sample collections (except the 1976 and 2000 universe collections)
2011 Universe (all public schools and districts; 99.2% of schools responded)
└── OCR 2011-12 Data Snapshot
2013 Universe (16,758 districts; 95,507 schools)
└── Official First Look
2015+ Universe collections (2015, 2017, 2020, 2021, 2023)
└── Definitions and topic availability still change by cycle
```
Every modern collection (2011-12, 2013-14, 2015-16, 2017-18, 2020-21, 2021-22, 2023-24) is a universe collection per OCR. Collections from 1968 to 2010 were samples, except 1976 and 2000. Years in this timeline are Portal year labels (fall year of the school year).
### Geographic Coverage
| Year | States | Territories |
|------|--------|-------------|
| 2011-15 | 50 states + DC | None |
| 2017+ | 50 states + DC | Puerto Rico |
### School Types Covered
| School Type | Coverage |
|-------------|----------|
| Traditional public | All years |
| Charter | All years |
| Magnet | All years |
| Alternative | All years |
| Virtual | 2015+ (expanded) |
| Justice facility | All years |
---
## Implications for Analysis
### Time Series Recommendations
| Analysis Period | Recommendation |
|-----------------|----------------|
| 2011-2013 | Both universe collections; harmonize definitions first |
| 2011-2017 | Universe throughout; check definitions and topic availability by cycle |
| 2015-2017 | Good for trends, except chronic absenteeism (CRDC form in 2015-16, EDFacts from 2017-18: a source change); add 2020 (2020-21) only with explicit COVID caveats |
| Including 2020-21 | Not recommended for trends |
| 2017 vs 2021 (2017-18, 2021-22; skip 2020) | Usable with caution: referrals to law enforcement and school-related arrests not directly comparable (2021-22 definition revisions); chronic absenteeism from EDFacts in both |
### Variable Availability Matrix
```python
def check_variable_availability(variable, years):
"""
Check which years a variable is available.
"""
# Portal year labels = fall year of the school year (2020 = 2020-21)
availability = {
'chronic_absenteeism': [2013, 2015, 2017, 2020, 2021],
'computer_science': [2017, 2020, 2021],
'suspension_instances': [2015, 2017, 2020, 2021],
'teacher_absenteeism': [2013, 2015, 2017, 2020, 2021],
'oss_total': [2011, 2013, 2015, 2017, 2020, 2021],
'restraint_physical': [2011, 2013, 2015, 2017, 2020, 2021],
}
var_years = availability.get(variable, [])
return {
'available_years': [y for y in years if y in var_years],
'missing_years': [y for y in years if y not in var_years]
}
```
```r
# Check which years a variable is available.
# Portal year labels = fall year of the school year (2020 = 2020-21)
availability <- list(
chronic_absenteeism = c(2013, 2015, 2017, 2020, 2021),
computer_science = c(2017, 2020, 2021),
suspension_instances = c(2015, 2017, 2020, 2021),
teacher_absenteeism = c(2013, 2015, 2017, 2020, 2021),
oss_total = c(2011, 2013, 2015, 2017, 2020, 2021),
restraint_physical = c(2011, 2013, 2015, 2017, 2020, 2021)
)
variable <- "chronic_absenteeism"
years <- c(2013, 2015, 2017)
# Unknown variables default to no available years
# (equivalent of Python's availability.get(variable, []))
var_years <- availability[[variable]]
if (is.null(var_years)) var_years <- integer(0)
available_years <- years[years %in% var_years]
missing_years <- years[!(years %in% var_years)]
```
### Comparing Across Years
**Safe Comparisons**:
- Same variable, same years (2015+)
- Relative measures (disparity ratios)
- Within-school changes over time
**Problematic Comparisons**:
- Pre-2015 vs. post-2015 (items optional in 2013-14 became required in 2015-16; coverage did not change)
- 2015 vs. 2017 chronic absenteeism (source change from the CRDC form to EDFacts)
- Variables that changed definition
- Including 2020-21 in trends
- Variables not collected all years
### Documentation Requirements
When using CRDC data across years, document:
1. **Years used** - Which collection years
2. **Coverage type** - Sample vs. universe
3. **Variable definitions** - For each year if changed
4. **Missing years** - If variable not available
5. **COVID exclusion** - If excluding 2020-21
6. **School panel** - If restricting to consistent schools
---
## Resources for Historical Research
### Official Documentation
| Resource | Content |
|----------|---------|
| Data element lists | Published for each collection |
| Collection forms | School and LEA forms by year |
| User manuals | Data file documentation |
| Codebooks | Variable definitions and codes |
### Where to Find Historical Documentation
- **ED.gov** - Official CRDC page, historical documents
- **CRDC Resource Center** - Collection materials
- **Education Data Portal mirrors** - Variable documentation via codebook `.xls` files (use `get_codebook_url()` from `fetch-patterns.md`)
- **NCES** - CRDC partnership documentation
- **Federal Register** - OMB clearance documents
> **Note:** When comparing variables across collection years, always verify definitions against the codebook for each specific year. The Portal codebook reflects the most recent collection; for earlier years, consult OCR's official data element lists.
### Version Tracking
Each CRDC collection has:
- OMB control number: 1870-0504
- Data element versions documented
- Form revision histories
- Public comment documentation
---
## Future Changes
### Anticipated Developments
| Potential Change | Status |
|------------------|--------|
| Nonbinary sex category | Removed for the 2025-26 CRDC |
| More granular disability categories | Possible |
| Behavioral intervention data | Possible |
| Reduced COVID variables | Expected |
| Additional course categories | Possible |
### Monitoring Changes
To stay current:
- Monitor Federal Register notices
- Review OMB clearance documents
- Check CRDC Resource Center
- Follow OCR announcements
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |