Skip to content

DAAF-Contribution-Community/daaf / .claude/skills/education-data-source-nccs

education-data-source-nccs

Verified

NCCS — Form 990 data for private nonprofit colleges (Portal: IPEDS-matched, 1993-2016). Revenue, expenses, assets, endowment, governance beyond IPEDS. Use when IRS financial depth needed. Portal ends 2016; public institutions excluded (no Form 990).

NCCS Data Source Reference

National Center for Charitable Statistics (NCCS) Form 990 data for private nonprofit colleges and universities (Portal mirror: IPEDS-matched institutions, 1993-2016). Use when IRS-based financial data — revenue, expenses, assets, endowment details — or governance information (board composition, executive compensation) is needed beyond what IPEDS provides. Portal data ends at 2016; for newer direct-source data, use the official NCCS Core catalog. As verified on 2026-10-19, the Core catalog lists a merged panel (1987-2024), SOI-current (2012-2024), and legacy (1987-2011) harmonized tiers plus a 'Classic NCCS format'; recent tax years are in progress as the IRS continues to release returns. Public institutions do not file Form 990 and are excluded.

The NCCS (National Center for Charitable Statistics) is the principal U.S. repository for empirical data on the nonprofit sector, derived from IRS Form 990 filings. It provides financial depth, governance detail, and historical coverage for private nonprofit colleges and universities that goes well beyond what IPEDS collects.

CRITICAL: Value Encoding

The Education Data Portal encodes ALL categorical variables as integers, not strings. Variable names are lowercase in Portal data. Always verify codes against codebooks.

Contextfipsmult_ein_flag
Portal integer6 (California)1 (Yes)
Original NCCS"06" or stringN/A

See ./references/variable-definitions.md for complete encoding tables.

What is NCCS?

  • Operator: Urban Institute, Center on Nonprofits and Philanthropy
  • Coverage: All U.S. tax-exempt nonprofit organizations (NCCS Unified BMF: 3,698,197 unique EINs, 1989–2026, build of 2026-09-15)
  • Primary data source: IRS Form 990 tax filings
  • Frequency: Annual (with filing lag)
  • Available years: Portal 1993-2016; direct merged Core 1987-2024; direct SOI-current 2012-2024; direct legacy 1987-2011 (recent tax years in progress as the IRS continues to release returns, catalog verified 2026-10-19)
  • Primary identifier: EIN (Employer Identification Number, 9-digit)
  • Education relevance: Private nonprofit colleges/universities are 501(c)(3) orgs filing Form 990; NTEE codes B40-B50 cover higher education

Reference File Structure

FilePurposeWhen to Read
nonprofit-data.mdNCCS datasets and what they containUnderstanding available data sources
form-990.mdIRS Form 990 structure and data elementsUnderstanding what information is collected
education-relevance.mdHow NCCS relates to higher educationConnecting nonprofit data to education research
ntee-codes.mdNonprofit classification systemFinding and filtering educational institutions
variable-definitions.mdKey financial and organizational variables, codes, special valuesInterpreting specific data elements or building queries

Decision Trees

What data am I looking for?

Research need?
├─ Financial data for private colleges → ./references/form-990.md
│   ├─ Revenue breakdown → Part VIII (Statement of Revenue)
│   ├─ Expenses by function → Part IX (Statement of Functional Expenses)
│   ├─ Assets and liabilities → Part X (Balance Sheet)
│   └─ Endowment details → Schedule D
├─ Governance/leadership → ./references/form-990.md
│   ├─ Board members → Part VII (Compensation)
│   ├─ Executive compensation → Part VII, Schedule J
│   └─ Policies and procedures → Part VI (Governance)
├─ Organizational characteristics → ./references/nonprofit-data.md
│   ├─ Basic info (name, address, EIN) → BMF (org_name_display, org_addr_*, ein / EIN2)
│   ├─ Tax-exempt type → BMF (subsection_code)
│   └─ NTEE classification → BMF (ntee_code_clean)
└─ Identify institutions → ./references/ntee-codes.md
    ├─ All higher education (NTEEV2 'UNI') → B40, B41, B42, B43, B50
    ├─ Universities → B43
    └─ Two-year / community colleges → B41

Which NCCS dataset should I use?

Dataset selection?
├─ Need universe of all nonprofits?
│   └─ Business Master File (BMF) → ./references/nonprofit-data.md
├─ Need detailed financial variables?
│   ├─ Large organizations (full 990 filers) → Core PC files
│   ├─ All organizations (990 + 990EZ) → Core PZ files
│   └─ Maximum detail (125 parsed tables) → Efile database
├─ Need private foundations?
│   └─ Core PF (Efile database has no 990-PF)
└─ Need small grassroots orgs?
    └─ 990-N ePostcard database

How do I connect NCCS to other education data?

Linking NCCS to education data?
├─ Need to match to IPEDS?
│   └─ See ./references/education-relevance.md (EIN-UNITID crosswalk)
├─ Need geographic analysis?
│   └─ Use BMF geocoded addresses + Census crosswalks
├─ Want institutional comparisons?
│   └─ Filter by NTEE codes B40-B50 for higher ed
└─ Analyzing trends over time?
    └─ Use the official Core catalog; choose merged Core (1987-2024) or SOI-current (2012-2024) and record in-progress status

Quick Reference: NCCS Datasets and Variables

Available Datasets

Portal mirror (via fetch_from_mirrors()):

DatasetDescriptionCoverageRows
990 Forms (nccs/colleges_nccs_all)Form 990 data for higher ed institutions matched to IPEDS1993-2016, ~2,600 institutions~30K

Direct NCCS sources (outside Portal):

DatasetDescriptionCurrent Catalog CoverageKey Use
Merged CoreHarmonized Form 990/990-EZ financial records1987-2024Broad historical financial analysis
SOI-current CoreCurrent-panel Core files2012-2024More recent annual analysis
Business Master File (BMF)Tax-exempt organization registryVerify in current BMF catalogSampling frame, basic information
IRS 990 EfileFull electronic filings with detailed fieldsVerify in current Efile catalogDetailed governance and programs

Direct-source status (verified 2026-10-19): As verified on 2026-10-19, the Core catalog lists a merged panel (1987-2024), SOI-current (2012-2024), and legacy (1987-2011) harmonized tiers plus a 'Classic NCCS format'; recent tax years are in progress as the IRS continues to release returns. Start at the official Core catalog, https://urbaninstitute.github.io/nccs/catalogs/catalog-core.html, and record the selected product, tax years, retrieval date, and in-progress status.

Scope note: The Portal mirror contains only Form 990 data matched to IPEDS institutions (~2,600 institutions), fixed at 1993-2016. It does NOT include the broader nonprofit universe or NTEE fields. For non-education nonprofits, NTEE-based filtering, or newer tax years, select the appropriate direct product from the official NCCS catalogs rather than treating "NCCS" as one undifferentiated dataset.

Key Identifiers

Portal dataset:

IDFormatLevelExampleNotes
unitidIntegerInstitution110635IPEDS institution ID (Portal addition)
einStringOrganization042103580Employer Identification Number (String, 9 characters with leading zeros, e.g. 042103580; verify mirror dtype and zero-pad to 9 before joining)
fipsIntegerState6 (California)State FIPS code

Full NCCS (direct download, outside Portal):

IDFormatLevelExampleNotes
ein / EIN2XX-XXXXXXX / EIN-XX-XXXXXXXOrganization04-2104327 / EIN-04-2104327ein in the Unified BMF and harmonized Core panel; EIN2 in the Unified BMF, Classic (HRMN) Core and Efile tables, so join those on EIN2. Raw legacy files, Efile ORG_EIN and the Portal ein use the 9-digit padded form (042104327): reformat before joining Portal data to the Unified BMF (NCCS EIN format convention)
ntee_code_cleanLetter + 2 digitsClassificationB42 (4-year college)Unified BMF standardized code (ntee_code_raw = as in the source file); may be imprecise (~25%) (NCCS, NTEE codes page: 'roughly one organization in four'); NOT in Portal data
subsection_code / subsection_cdcharacter / integerTax subsection"03" / 3 (501(c)(3) charity)subsection_code in the Unified BMF (character, keeps the leading zero); subsection_cd in the harmonized Core (integer); NOT in Portal data
geo_county_fips5-character stringGeography06037 (LA County)Not a Unified BMF column: join the geocoded BMF to the NCCS county FIPS crosswalk on (geo_state_abbr, geo_county) (NCCS BMF catalog); Portal uses state-level integer

Education NTEE Codes

CodeDescription
B20-B29Elementary & Secondary Schools
B40Higher Education Institutions (General)
B41Two-Year Colleges
B42Undergraduate Colleges (4-year)
B43Universities
B50Graduate/Professional Schools
B60Adult/Continuing Education
B70Libraries
B80Student Services/Organizations
B90Educational Services/Schools N.E.C.

Portal Variable Name Mapping (Selected)

The Portal dataset has 161 columns. Key Portal columns, the legacy NCCS Core name each one likely came from (the Portal does not document its sources), and the column to use in current direct NCCS harmonized Core files:

Portal NameLegacy NCCS Core name (pre-2026)Current harmonized Core name (basis)Description
year— (Portal-derived)—Academic year (fall semester) — Portal-aligned year (1993-2016)
fiscal_yearFISYR (fiscal year ending year)tax_year (DESC)IRS fiscal year ending year from 990 filing (1994-2017); typically year + 1
unitid——IPEDS institution ID (Portal addition)
einEINein, formatted XX-XXXXXXX (SV)Employer Identification Number (Portal: 9-digit padded)
fipsFIPSnone in Core (NF); BMF geography insteadState FIPS code (integer)
inst_name_nccsNAMEnone in harmonized Core (NF); Unified BMF org_name_displayOrganization name from 990 filing
mult_ein_flag——Multiple-EIN indicator (0=No, 1=Yes)
contributions_totalCONTtotal_contributions (SV)Total contributions
prog_serv_revPROGREVprogram_service_revenue (SV)Program service revenue
revenue_totalTOTREVtotal_revenue (SV)Total revenue
expenses_totalEXPStotal_expenses (SV)Total expenses
total_assets_eoyTOTASStotal_assets_eoy (LOC)Total assets (end of year)
net_assets_eoyNETASStotal_net_assets_eoy (LOC)Net assets (end of year)
compensation_officersCOMPENScompensation_current_officers (SV)Officer compensation
salaries_otherOTHSALother_salaries_wages (SV)Other salaries

Legacy names and basis tags. No current NCCS file uses the uppercase legacy names as column names; they survive only in the source_var column of the harmonized Core legacy-tier data dictionaries. SV = the legacy name is listed in that column's source_var; LOC = matched by the Form 990 line the dictionary gives; DESC = matched by description only (inference); NF = no current column found. Full mapping, including Classic (HRMN) and Efile names: ./references/variable-definitions.md.

Note: The full 161 columns include detailed revenue breakdowns (Part VIII), expense breakdowns (Part IX), and balance sheet items (Part X). Consult the codebook for the complete list. Use get_codebook_url("nccs/codebook_colleges_nccs_form_990") from fetch-patterns.md to download it.

Missing Data Codes

CodeMeaningWhen Used
-1MissingGeneric Portal label (not defined in NCCS source files)
-2Not applicableField doesn't apply to this entity
-3SuppressedGeneric Portal label (not defined in NCCS source files); rare in NCCS columns
nullNot reportedOrganization did not report (may have data)
0Zero valueExplicitly reported as zero

Important: In Portal data, -1/-2/-3 are integer values. However, empirically these codes are rare in the NCCS Portal dataset — most missing data appears as null rather than negative codes. Only a handful of financial columns (e.g., sale_sec_gross_net, changes_net_assets_other) contain any -1/-2/-3 values. A 0 means the organization reported zero; null means the organization did not report. These are distinct conditions.

Data Access

Datasets for NCCS are available via the mirror system. See datasets-reference.md for canonical paths, mirrors.yaml for mirror configuration, and fetch-patterns.md for fetch code patterns.

DatasetTypeYearsPathCodebook
990 FormsSingle1993-2016nccs/colleges_nccs_allnccs/codebook_colleges_nccs_form_990

Codebooks are .xls files co-located with data in all mirrors. Use get_codebook_url() from fetch-patterns.md to construct download URLs.

Truth Hierarchy: When interpreting variable values, apply this priority:

  1. Actual data file (what you observe in the parquet/CSV) — this IS the truth
  2. Live codebook (.xls in mirror) — authoritative documentation, may lag
  3. This skill documentation — convenient summary, may drift from codebook

If this documentation contradicts the codebook, trust the codebook. If the codebook contradicts observed data, trust the data and investigate.

Filtering

The Portal dataset is pre-filtered to higher education institutions matched to IPEDS UNITIDs. NTEE codes are NOT included in the Portal data — for NTEE-based filtering, use the full NCCS BMF directly.

import polars as pl

# Filter by year
df_2015 = df.filter(pl.col("year") == 2015)

# Filter by state (integer FIPS codes)
df_ca = df.filter(pl.col("fips") == 6)  # California

# Filter out null values from financial columns
df_valid = df.filter(
    (pl.col("revenue_total").is_not_null()) &
    (pl.col("revenue_total") >= 0)  # Excludes rare -1/-2/-3 codes
)

# Replace negative codes with null for analysis (precautionary)
df = df.with_columns(
    pl.when(pl.col("revenue_total") < 0)
    .then(None)
    .otherwise(pl.col("revenue_total"))
    .alias("revenue_total_clean")
)
library(dplyr)

# Filter by year
df_2015 <- df |> filter(year == 2015)

# Filter by state (integer FIPS codes)
df_ca <- df |> filter(fips == 6)  # California

# Filter out null values from financial columns
df_valid <- df |> filter(!is.na(revenue_total), revenue_total >= 0)

# Replace negative codes with NA for analysis (precautionary)
df <- df |> mutate(
  revenue_total_clean = if_else(revenue_total < 0, NA_real_, revenue_total)
)

Common Pitfalls

PitfallIssueSolution
Using string codesPortal uses integer FIPS (6), not string ("CA" or "06")Always use integer comparisons; filter >= 1 to exclude missing codes
Filing threshold confusionOrganizations with gross receipts < $200K and total assets < $500K may file 990-EZ with fewer variablesCheck form type; use Core PZ files for combined 990/990-EZ coverage
Fiscal year variationNonprofits have different fiscal year ends (June 30 common for colleges)year = Portal academic year (fall semester); fiscal_year = IRS fiscal year end (typically year + 1). Filter on year for IPEDS joins.
No NTEE in Portal dataPortal dataset does not include NTEE codes; pre-filtered to higher ed institutionsCannot filter by NTEE code; use full NCCS BMF for NTEE-based filtering
NTEE classification accuracy~25% of NTEE codes estimated to be imprecise (NCCS, NTEE codes page: 'roughly one organization in four')Use the IRS code as the source of truth and NCCS corrections as a supplement (NCCS NTEE page); in the Unified BMF the columns are ntee_code_raw / ntee_code_clean and nteev2
Consolidated filingsSome university systems file consolidated 990s covering multiple campusesCheck mult_ein_flag; one EIN may represent multiple institutions
Form version changesForm 990 was redesigned in 2008; variable definitions changedBe cautious comparing pre-2008 and post-2008 data for governance variables
Missing vs. zero0 means explicitly reported zero; null means not reportedDistinguish between zero-value and not-reported before aggregating

Key Differences: NCCS vs. IPEDS

AspectNCCS (Form 990)IPEDS
CoverageAll 501(c)(3) nonprofitsTitle IV institutions only
Reporting BasisIRS fiscal yearIPEDS survey cycles
Financial FrameworkNonprofit accounting (GAAP)Education-specific categories
GovernanceDetailed board/compensation dataLimited HR data
ProgramsMission statements, activitiesDegree programs, enrollment
IdentifierEINUNITID
Update FrequencyAnnual (with lag)Annual

Exploration Workflow

Using Portal Data (Recommended for Education Research)

  1. Fetch data via fetch_from_mirrors("nccs/colleges_nccs_all")

    • Data is pre-filtered to higher education institutions matched to IPEDS
    • Already includes unitid for IPEDS joining
    • 161 financial/organizational variables, 1993-2016
  2. Filter and clean

    • Filter by year, state (FIPS), or institution
    • Handle nulls (primary missing data form in Portal)
    • Check for rare negative codes (-1/-2/-3) in financial columns
  3. Link to IPEDS using unitid and year columns

    • Join with IPEDS directory, enrollment, finance, etc.
  4. Analyze — See variable definitions for meaning and limitations

Using Direct NCCS Sources (for Newer Years or Broader Nonprofit Research)

  1. Open the official catalog

    • Core catalog: https://urbaninstitute.github.io/nccs/catalogs/catalog-core.html
    • Use this stable catalog as the sole documented Core navigation route
  2. Choose the product deliberately

    • Merged Core for harmonized 1987-2024 coverage
    • SOI-current for the 2012-2024 current panel
    • Use the BMF or Efile catalog only when registry fields or filing-level detail is required
  3. Record release status before downloading

    • As verified on 2026-10-19, recent tax years are in progress as the IRS continues to release returns
    • The catalog also lists a legacy (1987-2011) tier and a 'Classic NCCS format'
    • Capture product name, tax years, retrieval date, and any catalog status note
  4. Identify and validate organizations

    • Filter BMF/NTEE fields (for example B40-B50 for higher education) only in direct products that actually include them
    • Verify 501(c)(3) status where relevant and retain EINs
    • Do not assume a direct NCCS record has a Portal unitid; build or validate an EIN-UNITID crosswalk separately
  5. Extract, clean, and analyze

    • Follow the exact file links and documentation in the current catalog
    • Merge BMF attributes only when needed and handle product-specific missing values
    • See variable definitions for interpretation and limitations

Related Data Sources

SourceRelationshipWhen to Use
education-data-source-ipedsComplementary institution dataJoin on EIN-UNITID crosswalk for enrollment, degrees, and education-specific financials
education-data-explorerParent discovery skillRouting questions to mirror dataset files
education-data-queryData fetchingDownloading parquet/CSV files

Topic Index

TopicReference File
BMF overview./references/nonprofit-data.md
Core data series./references/nonprofit-data.md
Efile database./references/nonprofit-data.md
Form 990 structure./references/form-990.md
Revenue variables./references/form-990.md
Expense variables./references/form-990.md
Balance sheet./references/form-990.md
Governance data./references/form-990.md
Schedule details./references/form-990.md
Linking to IPEDS./references/education-relevance.md
Private college identification./references/education-relevance.md
Supplementing education research./references/education-relevance.md
NTEE code structure./references/ntee-codes.md
Education NTEE codes./references/ntee-codes.md
NTEEV2 format./references/ntee-codes.md
Financial variables./references/variable-definitions.md
Variable naming conventions./references/variable-definitions.md
Data quality issues./references/variable-definitions.md
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 196kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 192kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 192kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 192kUpdated Claude CodeCodexVerifiedInstall