DAAF-Contribution-Community/daaf / .claude/skills/education-data-source-nhgis
education-data-source-nhgis
VerifiedNHGIS — census geography crosswalks via Portal: links schools (ncessch) and colleges (unitid) to census tracts, block groups, CBSAs, and regions (census-geography vintages 1990-2020; mirror file-year/linkage coverage 1980, 1984-2023). Portal provides geography linkage tables ONLY; browse census demographics in the public NHGIS Data Finder, then sign in with a free IPUMS NHGIS account to submit, manage, or download extracts. API extract requests also need an API key. Use for linking education or institutional data to census geography.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 23.6 KB |
| references/boundary-changes.md | markdown | 9.1 KB |
| references/data-access.md | markdown | 17.0 KB |
| references/geographic-units.md | markdown | 8.6 KB |
| references/school-geography-links.md | markdown | 14.5 KB |
| references/time-series.md | markdown | 10.9 KB |
| references/variable-catalog.md | markdown | 18.4 KB |
references/boundary-changes.md
View on GitHub# Census Boundary Changes Over Time
Understanding how census boundaries change is critical for valid longitudinal analysis.
## Why Boundaries Change
| Reason | Frequency | Example |
|--------|-----------|---------|
| Population growth | Each census | Tract split in suburban area |
| Population decline | Each census | Tracts merged in shrinking city |
| Boundary corrections | Each census | Align with new streets |
| Political changes | Varies | New county (rare), city annexation |
| Federal guidelines | Each census | Target population thresholds |
## Tract Changes: Patterns
### Tract Splits
When tract population exceeds threshold (~8,000), it's split:
```
1990: Tract 1001.00
↓
2000: Tract 1001.01, Tract 1001.02
↓
2010: Tract 1001.01, Tract 1001.02, Tract 1001.03 (1001.01 split again)
```
**Common in**: Fast-growing suburbs, exurban areas
**Implication**: Cannot directly compare 1990 Tract 1001.00 to any single 2010 tract.
### Tract Mergers
When population falls below threshold (~1,200), tracts may merge:
```
1990: Tract 2001.00, Tract 2002.00
↓
2000: Tract 2001.00 (absorbed 2002.00)
```
**Common in**: Declining central cities, rural areas
### Boundary Adjustments
Minor changes to align with:
- New streets
- Changed municipal boundaries
- Improved mapping accuracy
### Renumbering
Sometimes entire county's tracts are renumbered:
- New numbering scheme
- Accommodate splits/mergers
- Results in discontinuous tract histories
## Quantifying Changes: 1990-2020
**Net growth**: 62,276 census tracts and BNAs (1990, incl. Puerto Rico and Outlying Areas; GARM ch. 10) → 85,528 census tracts (2020, incl. Puerto Rico and Island Areas; Census 2020 tallies)
## Block Group Changes
Block groups change more than tracts:
- Smaller population thresholds
- More sensitive to population shifts
- First digit of block numbers indicates block group
## Block Changes
Blocks are completely redefined each census:
- Based on current street network
- No continuity assumed
- Must use crosswalks for any historical analysis
## County Changes (Rare)
Counties rarely change, but notable exceptions:
| Change | Year | Description |
|--------|------|-------------|
| Broomfield, CO | 2001 | Created from parts of 4 counties |
| Alaska regions | Various | Borough/census area changes |
| Virginia cities | Various | Independent city changes |
| Connecticut | 2022 | Counties replaced by planning regions |
### 2022 Connecticut County Changes
**Major change**: Census Bureau switched from 8 historical counties to 9 planning regions.
- Affects all Connecticut tract and block group codes, but not their land extents
- GEOID changed for every CT tract (new county code embedded)
- Pre-2022 to post-2022 comparisons need an ID crosswalk
NHGIS crosswalks cover this change only between different decades (e.g., 2012 to 2022 units); NHGIS has no 2020-to-2022 crosswalk and recommends CTData's 2020-2022 crosswalks or Geocorr 2022 for that pair.
## TIGER/Line Improvements
TIGER/Line files (source for boundary shapefiles) improved substantially:
| Era | Accuracy | Notes |
|-----|----------|-------|
| Pre-2008 | Lower | Based on 1990s digitization |
| 2008-2009 | Major improvement | GPS, local data integration |
| 2010+ | High | Continued refinement |
| 2020+ | Very high | Modern surveying |
**Implication**: Historical shapefiles may not align with recent ones even for unchanged boundaries.
### NHGIS Solutions
NHGIS provides multiple shapefile versions:
| Basis | Best For |
|-------|----------|
| 2000 TIGER/Line+ | Comparing pre-2010 data |
| 2008 TIGER/Line+ | Bridge between eras |
| 2010 TIGER/Line+ | 2010 and later comparisons |
| 2020 TIGER/Line+ | 2020 and later data |
## Impact on Education Research
### Scenario 1: School Neighborhood Change
**Problem**: Track demographics around a school from 1990-2020.
**Challenge**: School is in Tract 1234.00 in 1990; by 2020 that tract is split into 1234.01, 1234.02, 1234.03.
**Solutions**:
1. Use geographically standardized time series (all years to 2010 boundaries)
2. Use crosswalks to allocate all years to 2020 tracts
3. Aggregate to constant geography (e.g., county)
### Scenario 2: School District Demographics Over Time
**Problem**: Compare district demographics 2000 vs 2020.
**Challenge**: School district boundaries may also change; tracts within district change.
**Solutions**:
1. For tracts: Use crosswalks to standardize
2. For district boundaries: Obtain boundaries for each year; note changes
3. Alternative: Use SAIPE district-level data (available 1995+)
### Scenario 3: Historical Segregation Analysis
**Problem**: Measure segregation trends across tracts since 1970.
**Challenge**: Tract coverage incomplete before 1990; boundaries changed significantly.
**Solutions**:
1. Use NHGIS nominally integrated time series (with caveats)
2. Use standardized tables for 1990-2020
3. For pre-1990: Consider county-level analysis or accept metro-only tract coverage
## Detecting Boundary Changes
### Method 1: Compare Tract Counts
```python
import polars as pl
# If tract count in area changed, boundaries changed
tracts_1990 = df_1990.filter(pl.col("county_fips_geo") == 6037).select("tract").n_unique()
tracts_2020 = df_2020.filter(pl.col("county_fips_geo") == 6037).select("tract").n_unique()
# Illustrative (unsourced): any change in the count signals boundary changes
# 1990/2010/2020 files store county_fips_geo as 5-digit state+county (6037); the 2000 file does not
# (values 10-990, mirror probe 2026-10-27), so this filter matches nothing there — see SKILL.md Common Pitfalls
```
```r
library(dplyr)
# If tract count in area changed, boundaries changed
tracts_1990 <- df_1990 |> filter(county_fips_geo == 6037) |> distinct(tract) |> nrow()
tracts_2020 <- df_2020 |> filter(county_fips_geo == 6037) |> distinct(tract) |> nrow()
# Illustrative (unsourced): any change in the count signals boundary changes
# 1990/2010/2020 files store county_fips_geo as 5-digit state+county (6037); the 2000 file does not
# (values 10-990, mirror probe 2026-10-27), so this filter matches nothing there — see SKILL.md Common Pitfalls
```
### Method 2: Check Crosswalk
```python
import polars as pl
# Crosswalk shows splits/mergers (direct NHGIS download, not Portal data)
crosswalk = pl.read_csv("nhgis_tr1990_tr2010.csv")
# Tracts that split (1990 tract → multiple 2010 tracts)
splits = crosswalk.group_by("GJOIN1990").agg(pl.col("GJOIN2010").n_unique().alias("n_targets"))
split_tracts = splits.filter(pl.col("n_targets") > 1)
# Tracts that merged (multiple 1990 tracts → 1 2010 tract)
merges = crosswalk.group_by("GJOIN2010").agg(pl.col("GJOIN1990").n_unique().alias("n_sources"))
merged_tracts = merges.filter(pl.col("n_sources") > 1)
```
```r
library(readr)
library(dplyr)
# Crosswalk shows splits/mergers (direct NHGIS download, not Portal data)
crosswalk <- read_csv("nhgis_tr1990_tr2010.csv")
# Tracts that split (1990 tract -> multiple 2010 tracts)
splits <- crosswalk |>
group_by(GJOIN1990) |>
summarise(n_targets = n_distinct(GJOIN2010))
split_tracts <- splits |> filter(n_targets > 1)
# Tracts that merged (multiple 1990 tracts -> 1 2010 tract)
merges <- crosswalk |>
group_by(GJOIN2010) |>
summarise(n_sources = n_distinct(GJOIN1990))
merged_tracts <- merges |> filter(n_sources > 1)
```
### Method 3: Visual Inspection
Overlay shapefiles from different years in GIS:
- Areas with misalignment = boundary changes
- Useful for specific study areas
## Best Practices
### For Longitudinal Analysis
1. **Default**: Use geographically standardized tables when available
2. **Custom variables**: Use crosswalks to standardize
3. **Always**: Document which boundaries/year you're using
4. **Report**: Uncertainty bounds from standardization
### For Cross-Sectional Analysis
1. **Match vintages**: Use boundaries from same year as data
2. **Document**: Boundary vintage in methods
3. **Be aware**: 2020 data uses 2020 boundaries (not comparable to 2010 without adjustment)
### For Education Data Portal Integration
The Portal's NHGIS endpoints provide:
- School-to-tract links based on decennial census years
- Different years use different boundary vintages
When merging Portal data with direct NHGIS downloads:
- Verify tract IDs match expected format
- Use appropriate crosswalks for different years
## Reference: Major Boundary Events
| Event | Year | Scope | Notes |
|-------|------|-------|-------|
| Full U.S. tract/BNA coverage | 1990 | National | Counties without tracts had block numbering areas |
| TIGER accuracy improvement | 2008 | National | Shapefile alignment changed |
| Connecticut county dissolution | 2022 | Connecticut | Major ID changes |
| Alaska borough changes | Various | Alaska | Ongoing adjustments |
| Virginia independent cities | Various | Virginia | Cities separate from counties |
## Resources
| Resource | Content | URL |
|----------|---------|-----|
| NHGIS Crosswalks | Interpolation files | nhgis.org/geographic-crosswalks |
| Census Relationship Files | Official Census crosswalks | census.gov/geographies/reference-files |
| ACS Geography Changes | Annual change documentation | census.gov (by year) |
| NHGIS Revision History | Data update announcements | nhgis.org/revision-history |
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |