brycewang-stanford/Auto-Empirical-Research-Skills / skills/30-zirui-song-claude-skills/data-doc
data-doc
VerifiedDocument datasets, variables, sources, and merge keys for replication
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 4.0 KB |
SKILL.md
View on GitHub---
name: data-doc
description: Document datasets, variables, sources, and merge keys for replication
---
# Data Documentation Assistant
Help document datasets systematically for replication packages, co-author handoffs, and future reference.
## Dataset Documentation Template
When documenting a dataset, capture:
### 1. Dataset Overview
- **Name:** (e.g., `firm_year_panel.parquet`)
- **Location:** (relative path from project root)
- **Created by:** (script that generates it)
- **Last updated:**
- **Unit of observation:** (firm-year, loan-quarter, etc.)
- **N observations:**
- **Time coverage:**
### 2. Source Data
| Source | Access | Raw File | Notes |
|--------|--------|----------|-------|
| WRDS Compustat | Subscription | `raw/compustat_funda.csv` | Annual fundamentals |
| Revelio Labs | Licensed | `raw/revelio_positions.parquet` | Via BU server |
| Hand-collected | Manual | `raw/manual_coding.xlsx` | See coding protocol |
### 3. Key Variables
| Variable | Type | Description | Source | Notes |
|----------|------|-------------|--------|-------|
| `gvkey` | str | Compustat firm identifier | Compustat | Primary key |
| `fyear` | int | Fiscal year | Compustat | |
| `at` | float | Total assets ($ millions) | Compustat | Winsorized 1/99 |
| `treated` | int | =1 if treated firm | Constructed | See section 4 |
### 4. Variable Construction
For constructed/derived variables, document:
```
Variable: treated
Definition: =1 if firm received first PE investment in year t
Construction:
1. Merge PitchBook deals to Compustat on EIN
2. Keep first deal per firm
3. Flag year of first investment
Script: 2a_construct_treatment.py, lines 45-78
```
### 5. Sample Filters
Document all filters applied:
| Filter | Observations Dropped | Remaining |
|--------|---------------------|-----------|
| Raw data | - | 150,000 |
| Drop financials (SIC 6000-6999) | 25,000 | 125,000 |
| Require non-missing assets | 5,000 | 120,000 |
| Require 2+ years in panel | 10,000 | 110,000 |
### 6. Merge Keys
| Dataset A | Dataset B | Key(s) | Match Rate | Notes |
|-----------|-----------|--------|------------|-------|
| Compustat | CRSP | gvkey | 95% | Via CCM link table |
| Compustat | PitchBook | EIN | 72% | Manual cleaning needed |
| Revelio | Compustat | company_name | 68% | Fuzzy match, see script |
---
## Quick Commands
- **"document this dataset"** - Generate full template for a dataset
- **"variable list"** - Create variable table only
- **"merge documentation"** - Focus on merge keys and match rates
- **"sample flow"** - Generate sample filter table
- **"codebook"** - Formal codebook format for replication package
---
## Codebook Format (for Replication Packages)
```
================================================================================
CODEBOOK: firm_year_panel.dta
Generated: 2026-01-15
================================================================================
IDENTIFICATION
gvkey Compustat permanent firm identifier
fyear Fiscal year
OUTCOME VARIABLES
roa Return on assets = ni/at (winsorized 1/99)
investment Capex/lagged assets = capx/L.at
TREATMENT VARIABLES
post =1 for years after treatment
treated =1 for firms ever treated
treat_post Interaction: treated × post (DiD coefficient)
CONTROLS
size Log total assets = ln(at)
leverage Book leverage = (dltt+dlc)/at
mtb Market-to-book = (prcc_f×csho)/ceq
FIXED EFFECTS
ff48 Fama-French 48 industry classification
state State of incorporation
================================================================================
```
---
## Data Provenance Checklist
Before finalizing a dataset, verify:
- [ ] All source files documented with access instructions
- [ ] Variable definitions are unambiguous
- [ ] Sample filters documented with observation counts
- [ ] Merge match rates reported
- [ ] Winsorization/trimming documented
- [ ] Missing value treatment documented
- [ ] Script that creates dataset is identified
- [ ] Date of creation recorded
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |