brycewang-stanford/Auto-Empirical-Research-Skills / skills/17-DAAF-Contribution-Community-daaf/dot-claude/skills/education-data-source-ipeds
education-data-source-ipeds
VerifiedIPEDS — primary federal postsecondary data (~6,500 institutions, 1980-present): enrollment, completions, graduation rates, finance, aid, admissions, HR. For college/university analysis. Grad rates = first-time full-time; finance needs GASB/FASB care.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 23.7 KB |
| references/completions-data.md | markdown | 12.7 KB |
| references/data-quality.md | markdown | 13.4 KB |
| references/enrollment-data.md | markdown | 11.8 KB |
| references/finance-data.md | markdown | 15.1 KB |
| references/financial-aid.md | markdown | 18.2 KB |
| references/graduation-rates.md | markdown | 17.6 KB |
| references/institution-identifiers.md | markdown | 11.5 KB |
| references/survey-components.md | markdown | 25.6 KB |
references/financial-aid.md
View on GitHub# IPEDS Financial Aid Data
Understanding student financial aid data, net price calculations, and their limitations.
## Contents
- [Financial Aid Survey Overview](#financial-aid-survey-overview)
- [Student Populations](#student-populations)
- [Aid Types](#aid-types)
- [Net Price](#net-price)
- [Cost of Attendance](#cost-of-attendance)
- [Military Benefits](#military-benefits)
- [Common Analysis Issues](#common-analysis-issues)
- [Variable Reference](#variable-reference)
## Financial Aid Survey Overview
The Student Financial Aid (SFA) survey collects:
- Counts of students receiving different types of aid
- Total amounts awarded by aid type
- Average amounts calculated from these data
### Collection Period
Winter collection (December-February) for the prior academic year.
### Reporting Requirements
All Title IV institutions must report SFA data.
### Key Data Elements
1. Number of students receiving aid (by type)
2. Total amount of aid awarded (by type)
3. Average award = Total amount / Number receiving
## Student Populations
**Critical**: Different data elements use different student populations.
### Population Definitions
| Population | Definition | Primary Use |
|------------|------------|-------------|
| All undergraduates | All enrolled undergrads | Total aid picture |
| Full-time first-time (FTFT) | First-time, full-time, degree-seeking | Net price calculation |
| Full-time undergraduates | Full-time, degree-seeking | Comparative analysis |
| Part-time undergraduates | Part-time, degree-seeking | Limited reporting |
| Graduate students | Master's and doctoral | Separate from undergrad |
### Why Different Populations Matter
| Population | Pros | Cons |
|------------|------|------|
| All undergrads | Most comprehensive | Heterogeneous mix |
| FTFT only | Comparable across institutions | Small, selective group |
| Full-time only | More comparable | Misses PT students |
### First-Time Full-Time Limitation
Net price and many aid statistics use only FTFT students who:
- Never attended college before
- Started full-time
- Are degree/certificate-seeking
**At community colleges**, this may be <25% of students.
## Aid Types
### Grant Aid (Gift Aid)
Does not need to be repaid.
| Type | Source | Description |
|------|--------|-------------|
| Pell Grant | Federal | Need-based, up to ~$7,400/year |
| FSEOG | Federal | Supplemental grant, need-based |
| Other federal grants | Federal | Various programs |
| State grants | State | Varies by state |
| Local grants | Local | Varies by locality |
| Institutional grants | Institution | School-funded aid |
### Federal Loans
Must be repaid.
| Type | Description | Interest |
|------|-------------|----------|
| Direct Subsidized | Need-based, no interest while enrolled | Government pays |
| Direct Unsubsidized | Not need-based | Student pays |
| Direct PLUS | Parent loans for undergrads | Higher rate |
| Perkins | Campus-based (discontinued) | Low rate |
### Work-Study
Federal Work-Study program:
- Part-time employment
- Usually need-based
- Institution administers
### Aid Award Statistics
```python
# Average award calculation
avg_grant = total_grant_amount / number_receiving_grants
# Percent receiving aid
pct_receiving = students_with_aid / total_students * 100
```
## Net Price
### Definition
**Net Price = Cost of Attendance - Grant Aid Received**
This represents what students actually pay after grants (but before loans).
### Calculation Population
Net price is calculated ONLY for:
- First-time, full-time degree/certificate-seeking undergraduates
- Who were awarded any Title IV aid
**Excludes**:
- Part-time students
- Transfer students
- Students receiving no Title IV aid (full-pay students)
- Non-degree students
### Net Price by Income Level
IPEDS reports net price by family income quintile:
| Income Level | Income Range |
|--------------|--------------|
| $0 - $30,000 | Lowest income |
| $30,001 - $48,000 | Low-middle income |
| $48,001 - $75,000 | Middle income |
| $75,001 - $110,000 | Upper-middle income |
| $110,001 and above | Highest income |
### Why Net Price by Income Matters
```python
# Example showing why average net price is misleading
avg_net_price = 15000 # Overall average
# But by income level:
net_price_low_income = 8000 # After Pell, institutional aid
net_price_high_income = 22000 # Little grant aid
# Average masks important variation
```
### Net Price Limitations
| Limitation | Implication |
|------------|-------------|
| FTFT only | Not representative of all students |
| Title IV recipients only | Excludes full-pay students |
| Published vs actual | Individual packages vary widely |
| Excludes loans | Not total cost |
| Prior year data | May not reflect current prices |
### Interpreting Net Price
```python
# What net price tells you
net_price = sticker_price - grants
# What it doesn't tell you
# - Whether students can afford it
# - Loan amounts needed
# - Out-of-pocket costs
# - Impact of room and board choices
```
## Cost of Attendance
### Components
| Component | Description |
|-----------|-------------|
| Tuition and fees | Published tuition + required fees |
| Room and board | On-campus or estimated off-campus |
| Books and supplies | Estimated annual cost |
| Other expenses | Transportation, personal, etc. |
### Variations
| Student Type | Room/Board |
|--------------|------------|
| On-campus | On-campus rates |
| Off-campus (not with family) | Estimated local costs |
| Off-campus (with family) | Usually lower estimate |
### COA in Net Price Calculation
```python
# Net price components
coa = tuition_fees + room_board + books_supplies + other
net_price = coa - total_grant_aid
```
### Published vs Actual
| Measure | Description | Use |
|---------|-------------|-----|
| Published tuition | Sticker price | Comparison |
| Net tuition | After institutional discounts | Closer to reality |
| Net price | After all grants | What families pay |
| Out-of-pocket | After grants and loans | True cash needed |
## Military Benefits
### Types Tracked
| Benefit | Description |
|---------|-------------|
| Post-9/11 GI Bill | Veterans, service members |
| Yellow Ribbon | Institutional supplements |
| DoD Tuition Assistance | Active duty |
| Other military | Various programs |
### Data Collection
- Count of students receiving benefits
- Separate from other aid categories
- Graduate and undergraduate
### Limitation
Military benefits recipients may also receive other aid; avoid double-counting.
## Common Analysis Issues
### Issue 1: Comparing Net Price Across Institution Types
**Problem**: Different student populations make comparison misleading.
| Institution Type | Net Price Population |
|------------------|---------------------|
| Selective 4-year | FTFT is most students |
| Community college | FTFT is minority |
| For-profit | Variable enrollment patterns |
**Solution**: Note population differences; compare within peer groups.
### Issue 2: Average vs Distribution
**Problem**: Average net price hides important variation.
```python
# Hypothetical examples
# Institution A: Everyone pays $15,000
avg_a = 15000
# Institution B: Half pay $5,000, half pay $25,000
avg_b = 15000 # Same average, very different experience
```
**Solution**: Look at net price by income level.
### Issue 3: Full-Pay Students Not Included
**Problem**: Students not receiving Title IV aid are excluded.
| Group | Excluded Because |
|-------|-----------------|
| Wealthy families | Don't file FAFSA |
| International students | Not Title IV eligible |
| Undocumented students | Not Title IV eligible |
**Impact**: At wealthy institutions, net price may overstate what the full student body pays.
### Issue 4: Institutional Aid Variation
**Problem**: Institutional aid varies widely by student characteristics.
| Factor | Impact on Institutional Aid |
|--------|----------------------------|
| Academic merit | May get more aid |
| Athletic recruitment | May get more aid |
| Income level | Need-based aid varies |
| State residency | May affect aid |
**Solution**: Net price by income shows some of this variation.
### Issue 5: Debt vs Net Price
**Problem**: Low net price doesn't mean no debt.
```python
# Example
net_price = 12000 # After grants
pell_grant = 6000
institutional_grant = 4000
# Student needs $12,000 more
# Options: loans, work, family contribution
loans_taken = 8000 # Typical
out_of_pocket = 4000
```
**Solution**: Consider loan data alongside net price.
### Issue 6: Part-Time Students
**Problem**: Part-time students excluded from net price but:
- Are majority at many schools
- Have different aid patterns
- Face different cost structures
**Solution**: Note this limitation; use total aid data for broader picture.
## Data Interpretation Examples
### Comparing Two Institutions
```python
# Institution A (selective private)
net_price_a = 25000
pct_ftft = 80 # FTFT is 80% of students
interpretation: net_price reflects most students
# Institution B (community college)
net_price_b = 8000
pct_ftft = 20 # FTFT is 20% of students
interpretation: net_price reflects minority of students
```
### Analyzing Aid Effectiveness
```python
# Equity analysis
gap = net_price_high_income - net_price_low_income
# Positive gap = higher income pays more (progressive)
# Negative gap = lower income pays more (regressive)
# Zero = same net price regardless of income
```
### Grant Aid Coverage
```python
# What share of costs are covered by grants?
grant_coverage = total_grants / cost_of_attendance * 100
# High coverage = more affordable
# Low coverage = more loans/family contribution needed
```
## Variable Reference
> **IMPORTANT:** The Portal does NOT use NCES survey variable names (e.g., `scugffn`, `npist1`).
> Instead, the Portal restructures SFA data into **long/tidy format** with dimension columns
> (coded integers) and measure columns (counts, amounts, rates). NCES wide-format variables
> like `npist1`-`npist5` become rows distinguished by `income_level` codes 1-5.
>
> Multiple SFA-related datasets exist in the Portal:
> - `ipeds/colleges_ipeds_sfa_ftft` — FTFT student aid (1999-2017)
> - `ipeds/colleges_ipeds_sfa_grants_and_net_price` — Grants and net price (2008-2021)
> - `ipeds/colleges_ipeds_sfa_all_undergrads` — All undergrad aid (2007-2017)
> - `ipeds/colleges_ipeds_sfa_by_living_arrangement` — Aid by living arrangement (2008-2017)
> - `ipeds/colleges_ipeds_sfa_by_tuition_type` — Aid by tuition type (1999-2017)
>
> Use `get_codebook_url()` from `fetch-patterns.md` to download codebooks:
> ```python
> url = get_codebook_url("ipeds/codebook_colleges_ipeds_sfa_grants_and_net_price")
> ```
### Portal Column Reference by Dataset (Verified)
#### SFA FTFT (`sfa_ftft`) — 12 columns
| Portal Column | Type | Description |
|---------------|------|-------------|
| `unitid` | Int64 | Institution identifier |
| `year` | Int64 | Data year |
| `fips` | Int64 | State FIPS code |
| `type_of_aid` | Int64 | Aid type code (see coded values below) |
| `ftpt` | Int64 | Full-time/part-time status (coded) |
| `level_of_study` | Int64 | Level of study (coded) |
| `class_level` | Int64 | Class level (coded) |
| `degree_seeking` | Int64 | Degree-seeking status (coded) |
| `number_of_students` | Int64 | Count of students receiving this aid type |
| `percent_of_students` | Float64 | Proportion of students receiving aid |
| `average_amount` | Float64 | Average award amount (dollars) |
| `total_amount` | Float64 | Total amount awarded (dollars) |
**`type_of_aid` codes in this dataset:** 1, 2, 3, 4, 5, 7, 8, 10, 11, 12
> **Note on `type_of_aid` codes:** These are integer codes identifying aid categories. Definitions vary by dataset. In `sfa_grants_and_net_price`, code `9` = **all grant and scholarship aid** (Pell + institutional + state/local combined) -- this is NOT Pell-specific. Consult the codebook for the specific dataset you are using: `get_codebook_url("ipeds/codebook_colleges_ipeds_sfa_ftft")`.
#### SFA Grants and Net Price (`sfa_grants_and_net_price`) — 15 columns
| Portal Column | Type | Description |
|---------------|------|-------------|
| `unitid` | Int64 | Institution identifier |
| `year` | Int64 | Data year |
| `fips` | Int64 | State FIPS code |
| `type_of_aid` | Int64 | Aid type code (3 or 9 in this dataset) |
| `income_level` | Int64 | Income bracket code (see below) |
| `ftpt` | Int64 | Full-time/part-time status (coded) |
| `level_of_study` | Int64 | Level of study (coded) |
| `class_level` | Int64 | Class level (coded) |
| `degree_seeking` | Int64 | Degree-seeking status (coded) |
| `tuition_type` | Int64 | Tuition type (coded) |
| `number_of_students` | Int64 | Count of students in group |
| `number_receiving_grants` | Int64 | Count receiving grants |
| `total_grant` | Int64 | Total grant amount (dollars) |
| `average_grant` | Int64 | Average grant amount (dollars) |
| `net_price` | Int64 | Net price (COA minus grants, dollars) |
**`income_level` codes:** 1=$0-$30,000, 2=$30,001-$48,000, 3=$48,001-$75,000, 4=$75,001-$110,000, 5=$110,001+, 99=Total
#### SFA All Undergraduates (`sfa_all_undergrads`) — 10 columns
| Portal Column | Type | Description |
|---------------|------|-------------|
| `unitid` | Int64 | Institution identifier |
| `year` | Int64 | Data year |
| `fips` | Int64 | State FIPS code |
| `type_of_aid` | Int64 | Aid type code (3, 5, or 11 in this dataset) |
| `ftpt` | Int64 | Full-time/part-time status (coded) |
| `level_of_study` | Int64 | Level of study (coded) |
| `number_of_students` | Int64 | Count of students |
| `percent_of_students` | Float64 | Proportion of students |
| `average_amount` | Float64 | Average award amount (dollars) |
| `total_amount` | Float64 | Total amount awarded (dollars) |
#### SFA by Living Arrangement (`sfa_by_living_arrangement`) — 11 columns
| Portal Column | Type | Description |
|---------------|------|-------------|
| `unitid` | Int64 | Institution identifier |
| `year` | Int64 | Data year |
| `fips` | Int64 | State FIPS code |
| `type_of_aid` | Int64 | Aid type code |
| `living_arrangement` | Int64 | Living arrangement code |
| `ftpt` | Int64 | Full-time/part-time status (coded) |
| `level_of_study` | Int64 | Level of study (coded) |
| `class_level` | Int64 | Class level (coded) |
| `degree_seeking` | Int64 | Degree-seeking status (coded) |
| `tuition_type` | Int64 | Tuition type (coded) |
| `number_of_students` | Int64 | Count of students |
#### SFA by Tuition Type (`sfa_by_tuition_type`) — 12 columns
| Portal Column | Type | Description |
|---------------|------|-------------|
| `unitid` | Int64 | Institution identifier |
| `year` | Int64 | Data year |
| `fips` | Int64 | State FIPS code |
| `tuition_type` | Int64 | Tuition type code |
| `type_of_cohort` | Int64 | Cohort type code |
| `ftpt` | Int64 | Full-time/part-time status (coded) |
| `level_of_study` | Int64 | Level of study (coded) |
| `class_level` | Int64 | Class level (coded) |
| `degree_seeking` | Int64 | Degree-seeking status (coded) |
| `number_of_students` | Int64 | Count of students |
| `percent_of_cohort` | Float64 | Proportion of cohort |
| `percent_of_undergrads` | Float64 | Proportion of all undergrads |
### How NCES Variables Map to Portal Structure
The Portal converts NCES wide-format variables into long/tidy rows. For example:
**Student counts** — NCES has separate variables per aid type (`scugpel` for Pell, `scugfsl` for federal loans, etc.). The Portal stores all of these as `number_of_students` rows differentiated by `type_of_aid` code:
| NCES Variable | Portal Equivalent |
|---------------|-------------------|
| `scugffn` | `number_of_students` in `sfa_grants_and_net_price` (filtered by `type_of_aid`) |
| `scugrad` | `number_of_students` in `sfa_ftft` (filtered by `type_of_aid`) |
| `scugpel` | `number_of_students` in `sfa_ftft` where `type_of_aid` = Pell code |
| `scugfsl` | `number_of_students` in `sfa_ftft` where `type_of_aid` = federal loan code |
**Aid amounts** — similarly collapsed:
| NCES Variable | Portal Equivalent |
|---------------|-------------------|
| `upgrnta` | `average_amount` in `sfa_all_undergrads` (filtered by `type_of_aid`) |
| `upgrnt` | `total_amount` in `sfa_all_undergrads` (filtered by `type_of_aid`) |
| `uagrnta` | `average_amount` in `sfa_ftft` (filtered by `type_of_aid`) |
| `uagrntt` | `total_amount` in `sfa_ftft` (filtered by `type_of_aid`) |
**Net price by income** — NCES uses `npist1`-`npist5`; the Portal uses `net_price` column with `income_level` dimension:
| NCES Variable | Portal Equivalent |
|---------------|-------------------|
| `npist1` | `net_price` where `income_level` = 1 ($0-$30K) |
| `npist2` | `net_price` where `income_level` = 2 ($30K-$48K) |
| `npist3` | `net_price` where `income_level` = 3 ($48K-$75K) |
| `npist4` | `net_price` where `income_level` = 4 ($75K-$110K) |
| `npist5` | `net_price` where `income_level` = 5 ($110K+) |
| `npgrn1`-`npgrn5` | `number_of_students` where `income_level` = 1-5 |
### Querying Net Price by Income (Example)
```python
import polars as pl
MIRROR = "https://huggingface.co/datasets/brhkim/education_data_portal_mirror/resolve/main"
url = f"{MIRROR}/ipeds/colleges_ipeds_sfa_grants_and_net_price.parquet"
df = pl.read_parquet(url)
# Net price by income level for a specific institution and year
net_prices = (
df.filter(
(pl.col("unitid") == 166027) # Example: MIT
& (pl.col("year") == 2020)
& (pl.col("income_level").is_in([1, 2, 3, 4, 5]))
& (pl.col("type_of_aid") == 9)
)
.select("income_level", "net_price", "number_of_students")
.sort("income_level")
)
```
### Cost Variables
Cost of attendance components are in the **tuition/fees datasets**, not SFA datasets:
- `ipeds/colleges_ipeds_ay_tuition_fees` — Academic year tuition and fees
- `ipeds/colleges_ipeds_ay_room_board_other` — Room, board, and other expenses
Consult those codebooks for column names:
```python
url = get_codebook_url("ipeds/codebook_colleges_ipeds_ay_tuition_fees")
```
#### NCES Cost Variable Names (for reference only)
These appear in NCES documentation but are NOT used in the Portal:
| NCES Name | Description | Portal Dataset |
|-----------|-------------|----------------|
| `tuition2` | In-state tuition and fees | `ay_tuition_fees` |
| `tuition3` | Out-of-state tuition and fees | `ay_tuition_fees` |
| `roomamt` | Room charges | `ay_room_board_other` |
| `boardamt` | Board charges | `ay_room_board_other` |
| `bksupply` | Books and supplies estimate | `ay_room_board_other` |
| `rmbrdamt` | Room and board combined | `ay_room_board_other` |
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |