DAAF-Contribution-Community/daaf / .claude/skills/education-data-source-pseo
education-data-source-pseo
VerifiedPSEO — Census data linking graduates to employment via LEHD wage records. Earnings percentiles at 1/5/10 years post-graduation by institution, degree, and CIP. Use for graduate earnings analysis. Coverage changes by release; Census V4.14.1 (2026Q2, posted 2026-08-25) lists 1,117 institution rows and partners in 37 states plus D.C. (plus online-only WGU).
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 22.3 KB |
| references/earnings-data.md | markdown | 10.4 KB |
| references/geographic-flows.md | markdown | 10.5 KB |
| references/industry-flows.md | markdown | 12.6 KB |
| references/lehd-methodology.md | markdown | 9.7 KB |
| references/state-coverage.md | markdown | 8.3 KB |
| references/variable-definitions.md | markdown | 15.7 KB |
references/variable-definitions.md
View on GitHub# Variable Definitions
Complete reference for PSEO variables, codes, and status flags.
> **CRITICAL: Portal vs Census API Encoding**
>
> This document describes **Education Data Portal** integer encodings, which differ from Census API string codes. The Portal converts categorical variables to integers for consistency.
>
> | Context | Baccalaureate | Associates | Masters | Division Code |
> |---------|---------------|------------|---------|---------------|
> | **Portal (integers)** | `5` | `3` | `7` | `9` |
> | Census API (strings) | `05` | `03` | `07` | `9` |
>
> **Key Points:**
> - All variable names are **lowercase** in Portal data
> - `opeid` is an **8-character string** (zero-padded, e.g. `"00182600"`; may contain letters) in the 2001-2018 shards but **Int64** (leading zeros dropped, e.g. `365800`) in the 2019-2021 shards of the pinned mirror vintage; normalize before joining
> - `cipcode` is a **2-digit integer** (e.g., `11` for Computer Science)
> - `industry` is a **String**, not an integer (e.g., `"54"`, `"31-33"`)
> - Missing data codes: `-1` (missing), `-2` (not applicable), `-3` (suppressed) — but PSEO earnings columns were observed as `-1` only, never `-3` (see Missing Data Codes below)
>
> **Truth Hierarchy:** When interpreting variable values, apply this priority:
> 1. **Actual data file** (what you observe in the parquet) -- this IS the truth
> 2. **Live codebook** (`.xls` in mirror, via `get_codebook_url()` from `fetch-patterns.md`) -- authoritative documentation, may lag
> 3. **This skill documentation** -- convenient summary, may drift from codebook
>
> If this documentation contradicts the codebook or observed data, trust the higher-priority source and flag the discrepancy.
## Contents
- [Identifier Variables](#identifier-variables)
- [Earnings Variables](#earnings-variables-portal-schema)
- [Flow Variables](#flow-variables-portal-schema)
- [Degree Level Codes](#degree-level-codes-portal-integer-encoding)
- [CIP Codes](#cip-codes-portal-integer-encoding)
- [Institution Codes](#institution-codes-portal-integer-encoding)
- [Geography Codes](#geography-codes-portal-integer-encoding)
- [Aggregation Levels](#aggregation-levels)
## Identifier Variables
### Institution Identifiers (Portal Integer Encoding)
| Variable | Description | Type |
|----------|-------------|------|
| `unitid` | IPEDS Unit ID | Integer |
| `opeid` | Office of Postsecondary Education ID | String (8 characters, zero-padded) in 2001-2018 shards; Int64 (no leading zeros) in 2019-2021 shards |
| `fips` | State FIPS code of institution | Integer |
> **Note:** Portal stores `unitid` as an integer. `opeid` is an 8-character string in the 2001-2018 mirror shards and an Int64 without leading zeros in the 2019-2021 shards; the Census API uses `INSTITUTION` in the 8-character form (e.g., "00365800"). Normalize every shard with `pl.col("opeid").cast(pl.String).str.zfill(8)` (R: `stringr::str_pad(as.character(opeid), width = 8, pad = "0")`) before concatenating or joining; a relaxed concat otherwise leaves `"00365800"` and `"365800"` side by side.
>
> **Provenance (`opeid` dtype):** probed 2026-10-27 on the pinned mirror revision `0ad00ce0` (Portal v0.26.1). The schema is String in all 18 shards 2001-2018 (2018: 939 distinct opeid-unitid pairs, all 8 characters, one alphanumeric, `0033290W`) and Int64 in shards 2019-2021 (values 105100-30182251; UT Austin, unitid 228778, is `365800`). The skill's two example OPEIDs, `00182600` and `0033290W`, both carry `unitid` -1 (the Portal missing code) in the 2018 shard, so a unitid join or filter drops them. Maintainer-only run log, not shipped: `research/2026-09-18_AdHoc_Typesafe_Classifier_Experiments/probes/2026-10-27_mA_pseo_probe_a.py`. A Portal API probe of 2026-10-19 returned `"opeid":"00365800"` (UT Austin).
### Program Identifiers (Portal Integer Encoding)
| Variable | Description | Type |
|----------|-------------|------|
| `degree_level` | Type of credential | Integer (1-10) |
| `cipcode` | Classification of Instructional Programs | 2-digit integer |
### Cohort Identifiers
| Variable | Description | Type |
|----------|-------------|------|
| `pseo_cohort` | Graduation cohort | String (e.g., `"2016-2020"`, `"2019-2021"`) |
| `year` | Graduation year within the cohort span (Portal shard key). Each `pseo_cohort` row is repeated, with identical values, under every year in its span (e.g., cohort "2016-2018" appears under year 2016, 2017 and 2018); the Portal metadata label "Academic year (fall semester)" does not describe PSEO | Integer |
| `years_after_grad` | Years post-graduation | Integer (1, 5, or 10) |
## Earnings Variables (Portal Schema)
### Percentile Earnings
| Portal Variable | Description | Unit |
|-----------------|-------------|------|
| `p25_earnings` | 25th percentile earnings | 2022 dollars |
| `p50_earnings` | Median (50th percentile) earnings | 2022 dollars |
| `p75_earnings` | 75th percentile earnings | 2022 dollars |
| `years_after_grad` | Years post-graduation | `1`, `5`, or `10` |
> **Note:** Portal uses `years_after_grad` column to indicate timing. Census API embeds timing in variable names (Y1_*, Y5_*, Y10_*).
### Graduate Counts
| Portal Variable | Description |
|-----------------|-------------|
| `employed_grads_count_e` | Graduates with valid earnings |
| `total_grads_count` | Total IPEDS-reported graduates |
## Flow Variables (Portal Schema)
### Employment Counts
| Portal Variable | Description |
|-----------------|-------------|
| `employed_grads_count_f` | Employed graduates count |
| `employed_instate_grads_count` | Employed in institution's state |
| `jobless_m_emp_grads_count` | Non-employed or marginally employed |
### Industry Classification
| Portal Variable | Description |
|-----------------|-------------|
| `industry` | 2-digit NAICS sector (String, e.g., `"54"`, `"31-33"`, `"44-45"`) |
### Geography of Employment
| Portal Variable | Description |
|-----------------|-------------|
| `census_division` | Census Division code (1-9, 99 for aggregate) |
## Missing Data Codes (Portal Standard)
| Code | Meaning | When Used |
|------|---------|-----------|
| `-1` | Missing/not reported | Value unknown or not submitted |
| `-2` | Not applicable | Item doesn't apply to this record |
| `-3` | Suppressed | Data suppressed for privacy (Portal-wide code) |
> **PSEO observation (2026-10-19):** In the Portal probe for UT Austin, `year` 2016 (690 rows), every unavailable earnings value (`p25_earnings`/`p50_earnings`/`p75_earnings`, `employed_grads_count_e`) was `-1` and none was `-3`; `-3` appeared only in `jobless_m_emp_grads_count` on Census-division breakdown rows. Census suppresses earnings cells with protected counts below 30, so treat any negative earnings value as non-reportable; in that probe a `== -3` filter on earnings returns no rows.
### Handling Missing Data in Portal PSEO Data
```python
import polars as pl
# Load PSEO data
df = pl.read_parquet("colleges_pseo_2020.parquet")
# Filter out missing earnings data
valid_earnings = df.filter(pl.col("p50_earnings") > 0)
# Unavailable earnings (missing or suppressed): observed as -1 in PSEO, not -3
unavailable = df.filter(pl.col("p50_earnings") < 0)
# Convert coded missing to null
df_clean = df.with_columns(
pl.when(pl.col("p50_earnings") < 0)
.then(None)
.otherwise(pl.col("p50_earnings"))
.alias("p50_earnings_clean")
)
```
```r
library(arrow)
library(dplyr)
# Load PSEO data
df <- read_parquet("colleges_pseo_2020.parquet")
# Filter out missing earnings data
valid_earnings <- df |> filter(p50_earnings > 0)
# Unavailable earnings (missing or suppressed): observed as -1 in PSEO, not -3
unavailable <- df |> filter(p50_earnings < 0)
# Convert coded missing to NA
df_clean <- df |> mutate(
p50_earnings_clean = if_else(p50_earnings < 0, NA_real_, as.double(p50_earnings))
)
```
## Degree Level Codes (Portal Integer Encoding)
| Code | Degree Level | CIP Level | Cohort Years |
|------|--------------|-----------|--------------|
| `1` | Certificate (< 1 year) | 4-digit | 5 |
| `2` | Certificate (1-2 years) | 4-digit | 5 |
| `3` | Associate's | 4-digit | 5 |
| `4` | Certificate (2-4 years) | 4-digit | 5 |
| `5` | Bachelor's | 4-digit | 3 |
| `6` | Post-Bacc Certificate | 4-digit | 5 |
| `7` | Master's | 2-digit only | 5 |
| `8` | Post-Masters Certificate | 4-digit | 5 |
| `9` | Doctoral-Research/Scholarship | 2-digit only | 5 |
| `10` | Doctoral-Professional Practice | 4-digit | 5 |
| `-1` | Missing/not reported | — | — |
| `-2` | Not applicable | — | — |
| `-3` | Suppressed data | — | — |
**Notes:**
- Bachelor's (`5`) is the default if not specified
- Master's and Doctoral-Research only have 2-digit CIP codes
- Cohort years affects valid cohort values
> **Census API comparison:** Census uses string codes `01`-`08`, `17` (Doctoral - Research/Scholarship) and `18` (Doctoral - Professional Practice) per the LEHD public-use schema. Portal uses integers 1-10 (by label, Portal `9`/`10` correspond to Census `17`/`18`).
## CIP Codes (Portal Integer Encoding)
### CIP Level
| Level | Description | Portal Example |
|-------|-------------|----------------|
| 2-digit | Broad field | `11` = Computer and Information Sciences |
> **Note:** Portal data uses 2-digit integer CIP codes only (no decimal points).
>
> **Observed on the pinned mirror vintage (Portal v0.26.1, revision `0ad00ce0`, probed 2026-10-27):** `cipcode` is Int64 in all 21 shards (2001-2021). In the two shards tabulated (2018, 2021) it holds bare 2-digit family codes plus `99` (45 and 43 distinct values; no 4- or 6-digit values), so `cipcode == 11` matches rows (150,390 in the 2018 shard). IPEDS `completions-2digcip` stores the same family in a 6-digit style (`110000`), and the Portal-derived PSEO variable list (`education-data-explorer` `variable-dictionary-pseo.md`) also labels values that way, but the PSEO file does not use it. To join PSEO to IPEDS completions, map the family code (×10000, leaving `99`); this mapping is inferred from the two probes, not a tested join. IPEDS also carries a few programs under full 6-digit codes beside their family code (e.g. `220101` beside `220000`, `511201` beside `510000`), and those rows sit outside the family row: in the IPEDS 1991 and 2023 shards (total rows), `cipcode == 99` equals the family rows plus these 6-digit rows (probed 2026-10-28). Add them to their family before comparing totals; whether PSEO's family code covers the same programs was not probed. Maintainer-only run logs, not shipped: `research/2026-09-18_AdHoc_Typesafe_Classifier_Experiments/probes/2026-10-27_mA2_pseo_cipcode_probe.py`, `research/2026-09-18_AdHoc_Typesafe_Classifier_Experiments/probes/2026-10-28_mB2_ipeds_completions_totals_probe_a.py`.
### Common 2-Digit CIP Codes (Portal Integers)
| Code | Field |
|------|-------|
| `1` | Agriculture |
| `3` | Natural Resources and Conservation |
| `4` | Architecture |
| `5` | Area, Ethnic, Cultural, Gender Studies |
| `9` | Communication, Journalism |
| `10` | Communications Technologies |
| `11` | Computer and Information Sciences |
| `13` | Education |
| `14` | Engineering |
| `15` | Engineering Technologies |
| `16` | Foreign Languages, Literatures, Linguistics |
| `19` | Family and Consumer Sciences |
| `22` | Legal Professions and Studies |
| `23` | English Language and Literature |
| `24` | Liberal Arts and Sciences |
| `25` | Library Science |
| `26` | Biological and Biomedical Sciences |
| `27` | Mathematics and Statistics |
| `30` | Multi/Interdisciplinary Studies |
| `31` | Parks, Recreation, Leisure, Fitness |
| `38` | Philosophy and Religious Studies |
| `40` | Physical Sciences |
| `42` | Psychology |
| `43` | Homeland Security, Law Enforcement |
| `44` | Public Administration and Social Service |
| `45` | Social Sciences |
| `50` | Visual and Performing Arts |
| `51` | Health Professions |
| `52` | Business, Management, Marketing |
| `54` | History |
| `99` | All programs |
| `-1` | Missing/not reported |
| `-2` | Not applicable |
| `-3` | Suppressed data |
Full reference: [CIPCODE Labels (CSV)](https://lehd.ces.census.gov/data/schema/latest/label_cipcode.csv)
## Institution Codes (Portal Integer Encoding)
### Portal Format
In the Education Data Portal, `unitid` is an integer; `opeid` is an 8-character string in the 2001-2018 mirror shards and an Int64 in the 2019-2021 shards (pinned mirror vintage):
| Variable | Type | Example |
|----------|------|---------|
| `unitid` | Integer | `100751` (University of Alabama) |
| `opeid` | String (2001-2018) / Int64 (2019-2021) | `"00105100"` / `105100` |
> **Note:** The Census API uses 8-character OPEID strings (e.g., "00365800"); the Portal mirror matches that form only in its 2001-2018 shards. Zero-pad to 8 (see Institution Identifiers above) before joining.
### Example UNITID Values
| unitid | Institution |
|--------|-------------|
| `228778` | University of Texas at Austin |
| `139755` | Georgia Institute of Technology |
| `170976` | University of Michigan - Ann Arbor |
| `110635` | University of California - Berkeley |
### Finding Institution Codes
1. [All PSEO Institution Codes (CSV)](https://lehd.ces.census.gov/data/pseo/latest_release/all/pseo_all_institutions.csv)
2. [Complete Institution Labels (CSV)](https://lehd.ces.census.gov/data/schema/latest/label_institution.csv)
## Geography Codes (Portal Integer Encoding)
### State FIPS Codes (`fips`)
| Code | State | Code | State |
|------|-------|------|-------|
| `1` | Alabama | `27` | Minnesota |
| `4` | Arizona | `29` | Missouri |
| `8` | Colorado | `30` | Montana |
| `9` | Connecticut | `36` | New York |
| `11` | District of Columbia | `39` | Ohio |
| `13` | Georgia | `40` | Oklahoma |
| `15` | Hawaii | `41` | Oregon |
| `17` | Illinois | `42` | Pennsylvania |
| `18` | Indiana | `44` | Rhode Island |
| `19` | Iowa | `45` | South Carolina |
| `22` | Louisiana | `46` | South Dakota |
| `23` | Maine | `48` | Texas |
| `25` | Massachusetts | `49` | Utah |
| `26` | Michigan | `51` | Virginia |
| `-1` | Missing | `54` | West Virginia |
| | | `55` | Wisconsin |
| | | `56` | Wyoming |
> **Note:** Portal uses integers (e.g., `1` not `"01"`). Missing values use `-1`.
Full reference: [State FIPS Labels (CSV)](https://lehd.ces.census.gov/data/schema/latest/label_fipsnum.csv)
### Census Division Codes (`census_division`)
| Code | Division |
|------|----------|
| `1` | New England |
| `2` | Middle Atlantic |
| `3` | East North Central |
| `4` | West North Central |
| `5` | South Atlantic |
| `6` | East South Central |
| `7` | West South Central |
| `8` | Mountain |
| `9` | Pacific |
| `99` | All divisions (aggregate) |
Full reference: [Division Labels (CSV)](https://lehd.ces.census.gov/data/schema/latest/label_geography_division.csv)
## Aggregation Levels
### AGG_LEVEL_PSEO
The `AGG_LEVEL_PSEO` variable indicates the combination of dimensions in a tabulation:
| Code | Dimensions Included |
|------|---------------------|
| `38` | Institution + Degree + CIP + Cohort |
| Other codes | Various combinations |
Use to filter for specific aggregation patterns in bulk data.
### Default Values
In Portal data, there are no implicit defaults -- you must filter explicitly. For reference, the Census API defaults are:
| Census API Parameter | Default Value | Portal Equivalent |
|---------------------|---------------|-------------------|
| `DEGREE_LEVEL` | `05` (Bachelor's) | `degree_level == 5` |
| `CIP_LEVEL` | `2` | Portal uses 2-digit CIP only |
| `GRAD_COHORT_YEARS` | `3` (if Bachelor's) or `5` | Implicit in `pseo_cohort` string |
| `INST_LEVEL` | `I` (Individual institution) | Not applicable in Portal |
## Schema Reference Files
| File | URL |
|------|-----|
| Complete schema | `lehd.ces.census.gov/data/schema/latest/lehd_public_use_schema.html` |
| Earnings variables | `lehd.ces.census.gov/data/schema/latest/variables_pseoe.csv` |
| Flows variables | `lehd.ces.census.gov/data/schema/latest/variables_pseof.csv` |
| All labels | `lehd.ces.census.gov/data/schema/latest/` (browse directory) |
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 321k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 321k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 321k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 197k | Updated | Claude CodeCodex | Verified | Install |