Skip to content

brycewang-stanford/Auto-Empirical-Research-Skills / skills/17-DAAF-Contribution-Community-daaf/dot-claude/skills/education-data-source-crdc

education-data-source-crdc

Verified

CRDC — biennial OCR survey of all U.S. public schools (2011-2021). Discipline, course access, harassment, restraint/seclusion by race/sex/disability/EL. Use for civil rights and equity analysis. 2020-21 COVID-impacted; 2011-14 sampled, not universe.

references/historical-changes.md

View on GitHub
# CRDC Historical Changes

This reference documents how the Civil Rights Data Collection has evolved across collection years, including changes to coverage, variables, and definitions.

## Contents

- [Timeline Overview](#timeline-overview)
- [Pre-Modern CRDC (1968-2008)](#pre-modern-crdc-1968-2008)
- [Modern CRDC Era (2009-Present)](#modern-crdc-era-2009-present)
- [Year-by-Year Changes](#year-by-year-changes)
- [Variable Introduction Timeline](#variable-introduction-timeline)
- [Definition Changes](#definition-changes)
- [Coverage Evolution](#coverage-evolution)
- [Implications for Analysis](#implications-for-analysis)

---

## Timeline Overview

```
1968    First collection (Elementary and Secondary School Civil Rights Survey)
  ↓
1970s   Periodic collections, varying scope
  ↓
1980s   Continued periodic collection
  ↓
1990s   Collection continued
  ↓
2000    Last pre-modern collection
  ↓
2004    NCES partnership begins evaluation
  ↓
2009-10 Modern CRDC redesign begins
  ↓
2011-12 First modern CRDC (sampled)
  ↓
2013-14 Expanded sample
  ↓
2015-16 First near-universe collection
  ↓
2017-18 Full universe, expanded variables
  ↓
2020-21 COVID-impacted collection
  ↓
2021-22 Post-pandemic collection
  ↓
2023-24 Current collection cycle
```

---

## Pre-Modern CRDC (1968-2008)

### Original Purpose

The collection began in 1968 as the **Elementary and Secondary School Civil Rights Survey**:
- Created to monitor school desegregation
- Focused on race/ethnicity data
- Supported enforcement of Civil Rights Act of 1964

### Key Characteristics

| Attribute | Description |
|-----------|-------------|
| Name | Elementary and Secondary School Survey |
| Frequency | Irregular (not consistently biennial) |
| Coverage | Sampled districts |
| Focus | Desegregation, race data |
| Technology | Paper-based, later electronic |

### Limitations of Pre-Modern Data

- **Not comparable** to modern CRDC
- Different collection instruments
- Different variable definitions
- Different sampling approaches
- Limited electronic availability

---

## Modern CRDC Era (2009-Present)

### The 2009-10 Redesign

Major transformation of the collection:
- New collection system developed
- Expanded data elements
- Focus on educational opportunity
- NCES partnership for quality improvement

### Key Changes from Pre-Modern

| Aspect | Pre-Modern | Modern |
|--------|------------|--------|
| Focus | Desegregation | Educational opportunity |
| Collection | Paper/manual | Web-based |
| Variables | Limited | Extensive |
| Frequency | Irregular | Biennial |
| Coverage | Sample | Universe (from 2015) |

---

## Year-by-Year Changes

### 2011-12 Collection

**First modern CRDC**

| Aspect | Detail |
|--------|--------|
| Coverage | Sample (~7,000 LEAs, ~72,000 schools) |
| New elements | Restraint/seclusion, preschool discipline |
| Form structure | Redesigned school and LEA forms |
| Submission | Web-based system |

**Key data elements introduced**:
- Restraint and seclusion
- Preschool suspensions and expulsions
- Harassment/bullying categories
- Advanced course access
- Staff qualifications

### 2013-14 Collection

**Expanded sample**

| Aspect | Detail |
|--------|--------|
| Coverage | Larger sample (~16,500 LEAs) |
| Changes | Minor refinements to definitions |
| New elements | Some optional elements added |

**Key changes**:
- Expanded sample size
- Clarified definitions based on 2011-12 experience
- Added optional elements for certain data

### 2015-16 Collection

**Near-universe collection**

| Aspect | Detail |
|--------|--------|
| Coverage | Near-universe (~96,000 schools) |
| New elements | Chronic absenteeism, preschool expansion |
| Significance | First near-complete national data |

**Major additions**:
- **Chronic absenteeism** - First collection of this variable
- Expanded preschool data
- Sexual harassment of students
- Sports participation (Title IX)

**What this means**:
- First year suitable for national totals
- Baseline for time series
- Comprehensive school-level coverage

### 2017-18 Collection

**Full universe, major expansion**

| Aspect | Detail |
|--------|--------|
| Coverage | Full universe (~96,000 schools) |
| New elements | Computer science, expanded staffing |
| Puerto Rico | First inclusion |

**Major additions**:
- **Computer science offerings and enrollment**
- School resource officer data
- Teacher absenteeism data
- Puerto Rico schools (first time)

**Expanded elements**:
- More detailed staffing variables
- Additional course offerings
- Refined harassment categories

### 2020-21 Collection

**COVID-impacted year**

| Aspect | Detail |
|--------|--------|
| Coverage | Universe (~97,500 schools) |
| Special context | COVID-19 pandemic |
| New elements | COVID-specific variables |

**COVID-specific additions**:
- Remote instruction amount
- Student remote learning percentage
- Internet and device access

**Important notes**:
- Data reflects pandemic disruption
- Many schools operated remotely
- Discipline data artificially low
- **NOT COMPARABLE** to other years for trends

**New variables (permanent)**:
- Suspension instances (total count)
- School days missed due to suspension

### 2021-22 Collection

**Post-pandemic baseline**

| Aspect | Detail |
|--------|--------|
| Coverage | Universe (~98,000 schools) |
| Context | Return to in-person learning |
| Changes | COVID variables retained |

**Significance**:
- First "normal" year post-pandemic
- Better baseline for future comparisons
- COVID variables included for transition

### 2023-24 Collection

**Current collection**

| Aspect | Detail |
|--------|--------|
| Status | In progress |
| Expected changes | Minor refinements |
| COVID variables | Likely reduced |

---

## Variable Introduction Timeline

### Discipline Variables

| Variable | First Year |
|----------|------------|
| In-school suspension | 2011 |
| Out-of-school suspension | 2011 |
| Expulsion | 2011 |
| Referral to law enforcement | 2011 |
| School-related arrest | 2011 |
| Corporal punishment | 2011 |
| Preschool suspension | 2011 |
| Suspension instances | 2020 |
| School days missed | 2020 |

### Restraint and Seclusion

| Variable | First Year |
|----------|------------|
| Mechanical restraint | 2011 |
| Physical restraint | 2011 |
| Seclusion | 2011 |
| Instance counts | 2011 |

### Harassment

| Variable | First Year |
|----------|------------|
| Race/color/national origin harassment | 2011 |
| Sex-based harassment | 2011 |
| Disability harassment | 2011 |
| Sexual orientation harassment | 2015 |
| Religion harassment | 2015 |

### Chronic Absenteeism

| Variable | First Year |
|----------|------------|
| Chronic absenteeism (15+ days) | 2015 |

### Course Access

| Variable | First Year |
|----------|------------|
| AP course offerings | 2011 |
| AP enrollment by subject | 2011 |
| IB enrollment | 2011 |
| Gifted/talented | 2011 |
| Algebra I/II, Geometry | 2011 |
| Chemistry, Physics | 2011 |
| Computer science | 2017 |

### Staffing

| Variable | First Year |
|----------|------------|
| Teacher FTE | 2011 |
| Teacher experience | 2011 |
| Teacher certification | 2011 |
| Counselor FTE | 2011 |
| Nurse FTE | 2011 |
| School resource officers | 2017 |
| Teacher absenteeism | 2017 |

---

## Definition Changes

### Suspension Definition Evolution

| Year | Definition |
|------|------------|
| 2011-14 | Students receiving one or more out-of-school suspensions |
| 2015+ | Separate counts for "one" and "more than one" OSS |

### Harassment Categories

| Year | Categories |
|------|------------|
| 2011-14 | Race, sex, disability (3 categories) |
| 2015+ | Added sexual orientation, religion (5 categories) |

### Chronic Absenteeism

| Year | Definition |
|------|------------|
| Pre-2015 | Not collected |
| 2015+ | Students missing 15 or more school days |
| 2020-21 | Same definition but complicated by remote learning |

### Restraint Definition Clarification

| Year | Change |
|------|--------|
| 2011-14 | Initial definitions |
| 2015+ | Clarified exclusion of physical escort |
| 2017+ | Additional guidance on timeout vs. seclusion |

---

## Coverage Evolution

### Sample to Universe Transition

```
2011    Sample (~7,000 LEAs)
        └── Stratified sample, not generalizable
        
2013    Expanded Sample (~16,500 LEAs)
        └── Larger but still sampled
        
2015    Near-Universe (~96,000 schools)
        └── First comprehensive coverage
        
2017+   Full Universe
        └── All public schools required
```

### Geographic Coverage

| Year | States | Territories |
|------|--------|-------------|
| 2011-15 | 50 states + DC | None |
| 2017+ | 50 states + DC | Puerto Rico |

### School Types Covered

| School Type | Coverage |
|-------------|----------|
| Traditional public | All years |
| Charter | All years |
| Magnet | All years |
| Alternative | All years |
| Virtual | 2015+ (expanded) |
| Justice facility | All years |

---

## Implications for Analysis

### Time Series Recommendations

| Analysis Period | Recommendation |
|-----------------|----------------|
| 2011-2013 only | Can compare, but both sampled |
| 2011-2017 | Not recommended (sample vs. universe) |
| 2015-2017 only | Good for trends |
| 2015-2020 | Good, but note COVID |
| Including 2020-21 | Not recommended for trends |
| 2017-2021-22 | Good post-pandemic baseline |

### Variable Availability Matrix

```python
def check_variable_availability(variable, years):
    """
    Check which years a variable is available.
    """
    availability = {
        'chronic_absenteeism': [2015, 2017, 2020, 2021],
        'computer_science': [2017, 2020, 2021],
        'suspension_instances': [2020, 2021],
        'teacher_absenteeism': [2017, 2020, 2021],
        'oss_total': [2011, 2013, 2015, 2017, 2020, 2021],
        'restraint_physical': [2011, 2013, 2015, 2017, 2020, 2021],
    }
    
    var_years = availability.get(variable, [])
    return {
        'available_years': [y for y in years if y in var_years],
        'missing_years': [y for y in years if y not in var_years]
    }
```

### Comparing Across Years

**Safe Comparisons**:
- Same variable, same years (2015+)
- Relative measures (disparity ratios)
- Within-school changes over time

**Problematic Comparisons**:
- Pre-2015 vs. post-2015 (coverage change)
- Variables that changed definition
- Including 2020-21 in trends
- Variables not collected all years

### Documentation Requirements

When using CRDC data across years, document:

1. **Years used** - Which collection years
2. **Coverage type** - Sample vs. universe
3. **Variable definitions** - For each year if changed
4. **Missing years** - If variable not available
5. **COVID exclusion** - If excluding 2020-21
6. **School panel** - If restricting to consistent schools

---

## Resources for Historical Research

### Official Documentation

| Resource | Content |
|----------|---------|
| Data element lists | Published for each collection |
| Collection forms | School and LEA forms by year |
| User manuals | Data file documentation |
| Codebooks | Variable definitions and codes |

### Where to Find Historical Documentation

- **ED.gov** - Official CRDC page, historical documents
- **CRDC Resource Center** - Collection materials
- **Education Data Portal mirrors** - Variable documentation via codebook `.xls` files (use `get_codebook_url()` from `fetch-patterns.md`)
- **NCES** - CRDC partnership documentation
- **Federal Register** - OMB clearance documents

> **Note:** When comparing variables across collection years, always verify definitions against the codebook for each specific year. The Portal codebook reflects the most recent collection; for earlier years, consult OCR's official data element lists.

### Version Tracking

Each CRDC collection has:
- OMB control number: 1870-0504
- Data element versions documented
- Form revision histories
- Public comment documentation

---

## Future Changes

### Anticipated Developments

| Potential Change | Status |
|------------------|--------|
| Gender identity data | Under consideration |
| More granular disability categories | Possible |
| Behavioral intervention data | Possible |
| Reduced COVID variables | Expected |
| Additional course categories | Possible |

### Monitoring Changes

To stay current:
- Monitor Federal Register notices
- Review OMB clearance documents
- Check CRDC Resource Center
- Follow OCR announcements
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall