Skip to content

brycewang-stanford/Auto-Empirical-Research-Skills / skills/29-quarcs-lab-project20XXy/dot-claude/skills/data-audit

data-audit

Verified

Scans notebooks for data file references and verifies each file exists on disk. Use when checking for broken data paths.

FileKindSize
SKILL.mdskill md1.8 KB
---
name: data-audit
description: Scans notebooks for data file references and verifies each file exists on disk. Use when checking for broken data paths.
allowed-tools: Bash, Read, Glob, Grep
---

# Audit Data References

Scan all notebooks for data file references and verify they exist on disk.

## Steps

1. Scan all `.ipynb` files in `notebooks/` for data loading patterns:
   - **Python:** `pd.read_csv(...)`, `pd.read_stata(...)`, `pd.read_excel(...)`, `pd.read_parquet(...)`, `open(...)`, `np.loadtxt(...)`
   - **R:** `read.csv(...)`, `read_csv(...)`, `read.dta(...)`, `haven::read_dta(...)`, `readxl::read_excel(...)`, `load(...)`
   - **Stata:** `use "..."`, `import delimited "..."`, `import excel "..."`, `insheet using "..."`
   - Also check the `.md` Jupytext pairs for the same patterns

2. Extract every referenced file path and normalize it:
   - Resolve relative paths from the notebook's directory (`notebooks/`)
   - Resolve paths using `DATA_DIR`, `RAW_DATA_DIR` from `config.py` / `config.R`

3. Check that each referenced file exists in `data/rawData/` or `data/`

4. Scan `data/rawData/` and `data/` for all data files present on disk

5. Report three categories:

   **Resolved — referenced and found:**
   - File path, which notebook references it, line/cell number

   **Broken — referenced but not found:**
   - File path as written in code, which notebook, suggested fix (closest matching file, or note that it may need to be downloaded)

   **Undocumented — on disk but never referenced by any notebook:**
   - File path in `data/rawData/` or `data/` that no notebook loads

6. Print a summary: total references, resolved, broken, undocumented files

## Error handling

- If no notebooks exist, report "No notebooks found" and stop.
- If `data/rawData/` does not exist, warn but continue checking `data/`.
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 196kUpdated Claude CodeCodexVerifiedInstall