Skip to content

mikeparcewski/wicked-garden / skills/data

wicked-garden-data

Verified

Data engineering, analysis, and ML toolkit. One skill, routed sub-actions: analyze (interactive EDA on a CSV/Excel/data file), profile / validate / quality (schema-level data engineering ops), ml review / ml pipeline (model review + training-pipeline design), pipeline design / pipeline review (ETL architecture), ontology (recommend Schema.org, Dublin Core, DCAT, FOAF, GoodRelations, SKOS, or a custom shape for a dataset), shapes (a live source's data shapes, proposed for review). Use when: "analyze this CSV/Excel/data file" / "explore this dataset" / one-off data exploration; "profile this dataset" / "validate data against a schema" / "generate a data quality report"; "review this ML model" / "design an ML training pipeline"; "design a data pipeline" / "review this ETL"; "recommend an ontology for this data" / "map columns to a public ontology"; "propose data shapes for <source>"; or any former /wicked-garden:data:* invocation. Delegated deep passes: the `wicked-garden-data-engineer` worker skill.

FileKindSize
SKILL.mdskill md8.0 KB
refs/analysis-templates.mdmarkdown3.3 KB
refs/analyze.mdmarkdown5.1 KB
refs/data-examples.mdmarkdown4.9 KB
refs/data.mdmarkdown3.8 KB
refs/ml-templates.mdmarkdown4.0 KB
refs/ml.mdmarkdown5.7 KB
refs/pipeline-templates.mdmarkdown4.6 KB
refs/pipeline.mdmarkdown4.8 KB
refs/shapes.mdmarkdown6.1 KB

refs/data.md

View on GitHub
# Data Engineering Rubric — profile / validate / quality

Apply this inline for schema-level checks. Caller has pre-read the data file
head/tail and captured columns / types / nulls.

## profile

Emit a structured profile with shape, per-column stats, and prioritized findings.

```markdown
## Data Profile: {name}

**Rows**: {count}  **Columns**: {count}  **File**: {path}

### Column Summary
| Column | Type | Null% | Unique | Min | Max | Sample |
|--------|------|-------|--------|-----|-----|--------|

### Key Observations
1. {High null-rate columns}
2. {Type anomalies or mixed types}
3. {Skew / outlier signals}

### Recommendations
1. {Prioritized action — e.g. handle nulls in col X}

**Confidence**: {HIGH|MEDIUM|LOW}
```

## validate

Compare actual data against the provided schema. Report breaking vs non-breaking drift.

```markdown
## Schema Validation: {name}

**Schema**: {schema-path}  **Data**: {data-path}

### Result: {PASS|FAIL}

### Breaking Issues (must fix)
| Column | Expected | Actual | Impact |
|--------|----------|--------|--------|

### Warnings (non-breaking)
| Column | Note |
|--------|------|

### Recommendations
1. {Fix breaking issues first}

**Confidence**: {HIGH|MEDIUM|LOW}
```

Schema design principles to apply:
- Explicit types (avoid variant unless necessary).
- Clear nullability contracts.
- Consistent naming conventions.
- Version schemas explicitly.
- Document breaking vs non-breaking changes.

## quality

Five-dimension quality assessment: Completeness, Uniqueness, Validity, Consistency, Timeliness.

```markdown
## Data Quality Report

**Dataset**: {name}  **Rows**: {count}  **Columns**: {count}

### Quality Metrics
| Dimension    | Score | Issues |
|--------------|-------|--------|
| Completeness | {%}   | {null columns} |
| Uniqueness   | {%}   | {duplicate rate} |
| Validity     | {%}   | {constraint violations} |
| Consistency  | {%}   | {cross-field failures} |
| Timeliness   | {age} | {freshness status} |

### Critical Issues
- {Issue with severity and impact}

### Recommendations
1. {Prioritized action items}

**Confidence**: {HIGH|MEDIUM|LOW}
```

## Quality thresholds

| Dimension | Metric | Threshold |
|-----------|--------|-----------|
| Completeness | Null rate | <5% |
| Uniqueness | Duplicate rate | <1% |
| Validity | Type conformance | 100% |
| Consistency | Cross-field rules | 100% |

## Scripted paths (optional)

Profile a dataset with `data_profiler.py`:

```bash
wicked-garden run scripts/data/data_profiler.py \
  --input data.csv --output profile.json
```

Output includes: row and column counts, inferred column types, null rates per
column, cardinality (distinct values), sample values, and statistical
summaries for numeric columns.

Validate against a schema with `schema_validator.py`:

```bash
wicked-garden run scripts/data/schema_validator.py \
  --schema schemas/expected.json \
  --data data/actual.csv
```

Schema columns define: **name**, **type** (integer, string, decimal, datetime,
date), **nullable** (true/false), **constraints** (unique, min, max, pattern,
enum). See [data-examples.md](data-examples.md) for the schema format, profile
output, quality-report example, common workflows, and SQL quality queries.

## Large files (>1GB)

Route to the `analyze` sub-action of the wicked-garden-data skill for
efficient SQL-based profiling via DuckDB.

## Integration

- Native tasks: document quality issues via TaskCreate with `metadata.event_type="task"`.
- wicked-garden-mem (store action): store quality patterns across sessions.

## General engineering checklist

- [ ] Schema-first: define schemas before processing.
- [ ] Fail fast: validate early, fail loudly.
- [ ] Idempotent: pipelines rerunnable without side effects.
- [ ] Observable: emit metrics + logs at every stage.
- [ ] Tested: data quality tests are non-negotiable.
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall