brycewang-stanford/Auto-Empirical-Research-Skills / skills/17-DAAF-Contribution-Community-daaf/dot-claude/skills/education-data-source-nhgis
education-data-source-nhgis
VerifiedNHGIS — census geography crosswalks via Portal: links schools (ncessch) and colleges (unitid) to tracts, block groups, CBSAs (1990-2020). Census demographics NOT in Portal — access NHGIS directly. Use for linking education data to census geography.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 18.5 KB |
| references/boundary-changes.md | markdown | 7.8 KB |
| references/data-access.md | markdown | 11.5 KB |
| references/geographic-units.md | markdown | 8.0 KB |
| references/school-geography-links.md | markdown | 11.0 KB |
| references/time-series.md | markdown | 9.3 KB |
| references/variable-catalog.md | markdown | 17.4 KB |
references/boundary-changes.md
View on GitHub# Census Boundary Changes Over Time
Understanding how census boundaries change is critical for valid longitudinal analysis.
## Why Boundaries Change
| Reason | Frequency | Example |
|--------|-----------|---------|
| Population growth | Each census | Tract split in suburban area |
| Population decline | Each census | Tracts merged in shrinking city |
| Boundary corrections | Each census | Align with new streets |
| Political changes | Varies | New county (rare), city annexation |
| Federal guidelines | Each census | Target population thresholds |
## Tract Changes: Patterns
### Tract Splits
When tract population exceeds threshold (~8,000), it's split:
```
1990: Tract 1001.00
↓
2000: Tract 1001.01, Tract 1001.02
↓
2010: Tract 1001.01, Tract 1001.02, Tract 1001.03 (1001.01 split again)
```
**Common in**: Fast-growing suburbs, exurban areas
**Implication**: Cannot directly compare 1990 Tract 1001.00 to any single 2010 tract.
### Tract Mergers
When population falls below threshold (~1,200), tracts may merge:
```
1990: Tract 2001.00, Tract 2002.00
↓
2000: Tract 2001.00 (absorbed 2002.00)
```
**Common in**: Declining central cities, rural areas
### Boundary Adjustments
Minor changes to align with:
- New streets
- Changed municipal boundaries
- Improved mapping accuracy
### Renumbering
Sometimes entire county's tracts are renumbered:
- New numbering scheme
- Accommodate splits/mergers
- Results in discontinuous tract histories
## Quantifying Changes: 1990-2020
| Transition | Approximate Scale |
|------------|-------------------|
| 1990 to 2000 | ~15% of tracts changed |
| 2000 to 2010 | ~18% of tracts changed |
| 2010 to 2020 | ~20% of tracts changed |
**Net growth**: ~62,000 tracts (1990) → ~74,000 tracts (2010) → ~85,000 tracts (2020)
## Block Group Changes
Block groups change more than tracts:
- Smaller population thresholds
- More sensitive to population shifts
- First digit of block numbers indicates block group
## Block Changes
Blocks are completely redefined each census:
- Based on current street network
- No continuity assumed
- Must use crosswalks for any historical analysis
## County Changes (Rare)
Counties rarely change, but notable exceptions:
| Change | Year | Description |
|--------|------|-------------|
| Broomfield, CO | 2001 | Created from parts of 4 counties |
| Alaska regions | Various | Borough/census area changes |
| Virginia cities | Various | Independent city changes |
| Connecticut | 2022 | Counties replaced by planning regions |
### 2022 Connecticut County Changes
**Major change**: Census Bureau switched from 8 historical counties to 9 planning regions.
- Affects all Connecticut tract and block group codes
- GEOID changed for every CT tract (new county code embedded)
- Requires crosswalks for any pre-2022 to post-2022 comparison
NHGIS provides crosswalks for this change in the 2022 non-census year crosswalks.
## TIGER/Line Improvements
TIGER/Line files (source for boundary shapefiles) improved substantially:
| Era | Accuracy | Notes |
|-----|----------|-------|
| Pre-2008 | Lower | Based on 1990s digitization |
| 2008-2009 | Major improvement | GPS, local data integration |
| 2010+ | High | Continued refinement |
| 2020+ | Very high | Modern surveying |
**Implication**: Historical shapefiles may not align with recent ones even for unchanged boundaries.
### NHGIS Solutions
NHGIS provides multiple shapefile versions:
| Basis | Best For |
|-------|----------|
| 2000 TIGER/Line+ | Comparing pre-2010 data |
| 2008 TIGER/Line+ | Bridge between eras |
| 2010 TIGER/Line+ | 2010 and later comparisons |
| 2020 TIGER/Line+ | 2020 and later data |
## Impact on Education Research
### Scenario 1: School Neighborhood Change
**Problem**: Track demographics around a school from 1990-2020.
**Challenge**: School is in Tract 1234.00 in 1990; by 2020 that tract is split into 1234.01, 1234.02, 1234.03.
**Solutions**:
1. Use geographically standardized time series (all years to 2010 boundaries)
2. Use crosswalks to allocate all years to 2020 tracts
3. Aggregate to constant geography (e.g., county)
### Scenario 2: School District Demographics Over Time
**Problem**: Compare district demographics 2000 vs 2020.
**Challenge**: School district boundaries may also change; tracts within district change.
**Solutions**:
1. For tracts: Use crosswalks to standardize
2. For district boundaries: Obtain boundaries for each year; note changes
3. Alternative: Use SAIPE district-level data (available 1995+)
### Scenario 3: Historical Segregation Analysis
**Problem**: Measure segregation trends across tracts since 1970.
**Challenge**: Tract coverage incomplete before 1990; boundaries changed significantly.
**Solutions**:
1. Use NHGIS nominally integrated time series (with caveats)
2. Use standardized tables for 1990-2020
3. For pre-1990: Consider county-level analysis or accept metro-only tract coverage
## Detecting Boundary Changes
### Method 1: Compare Tract Counts
```python
import polars as pl
# If tract count in area changed, boundaries changed
tracts_1990 = df_1990.filter(pl.col("county_fips_geo") == 6037).select("tract").n_unique()
tracts_2020 = df_2020.filter(pl.col("county_fips_geo") == 6037).select("tract").n_unique()
# LA County: ~1,600 in 1990 → ~2,300 in 2020
```
### Method 2: Check Crosswalk
```python
import polars as pl
# Crosswalk shows splits/mergers (direct NHGIS download, not Portal data)
crosswalk = pl.read_csv("nhgis_tr1990_tr2010.csv")
# Tracts that split (1990 tract → multiple 2010 tracts)
splits = crosswalk.group_by("GJOIN1990").agg(pl.col("GJOIN2010").n_unique().alias("n_targets"))
split_tracts = splits.filter(pl.col("n_targets") > 1)
# Tracts that merged (multiple 1990 tracts → 1 2010 tract)
merges = crosswalk.group_by("GJOIN2010").agg(pl.col("GJOIN1990").n_unique().alias("n_sources"))
merged_tracts = merges.filter(pl.col("n_sources") > 1)
```
### Method 3: Visual Inspection
Overlay shapefiles from different years in GIS:
- Areas with misalignment = boundary changes
- Useful for specific study areas
## Best Practices
### For Longitudinal Analysis
1. **Default**: Use geographically standardized tables when available
2. **Custom variables**: Use crosswalks to standardize
3. **Always**: Document which boundaries/year you're using
4. **Report**: Uncertainty bounds from standardization
### For Cross-Sectional Analysis
1. **Match vintages**: Use boundaries from same year as data
2. **Document**: Boundary vintage in methods
3. **Be aware**: 2020 data uses 2020 boundaries (not comparable to 2010 without adjustment)
### For Education Data Portal Integration
The Portal's NHGIS endpoints provide:
- School-to-tract links based on decennial census years
- Different years use different boundary vintages
When merging Portal data with direct NHGIS downloads:
- Verify tract IDs match expected format
- Use appropriate crosswalks for different years
## Reference: Major Boundary Events
| Event | Year | Scope | Notes |
|-------|------|-------|-------|
| Full U.S. tract coverage | 1990 | National | First complete tract system |
| TIGER accuracy improvement | 2008 | National | Shapefile alignment changed |
| Connecticut county dissolution | 2022 | Connecticut | Major ID changes |
| Alaska borough changes | Various | Alaska | Ongoing adjustments |
| Virginia independent cities | Various | Virginia | Cities separate from counties |
| Post-Katrina tract changes | 2006 | Louisiana | Special census area changes |
## Resources
| Resource | Content | URL |
|----------|---------|-----|
| NHGIS Crosswalks | Interpolation files | nhgis.org/geographic-crosswalks |
| Census Relationship Files | Official Census crosswalks | census.gov/geographies/reference-files |
| ACS Geography Changes | Annual change documentation | census.gov (by year) |
| NHGIS Revision History | Data update announcements | nhgis.org/revision-history |
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |