mikeparcewski/wicked-garden / skills/data
wicked-garden-data
VerifiedData engineering, analysis, and ML toolkit. One skill, routed sub-actions: analyze (interactive EDA on a CSV/Excel/data file), profile / validate / quality (schema-level data engineering ops), ml review / ml pipeline (model review + training-pipeline design), pipeline design / pipeline review (ETL architecture), ontology (recommend Schema.org, Dublin Core, DCAT, FOAF, GoodRelations, SKOS, or a custom shape for a dataset), shapes (a live source's data shapes, proposed for review). Use when: "analyze this CSV/Excel/data file" / "explore this dataset" / one-off data exploration; "profile this dataset" / "validate data against a schema" / "generate a data quality report"; "review this ML model" / "design an ML training pipeline"; "design a data pipeline" / "review this ETL"; "recommend an ontology for this data" / "map columns to a public ontology"; "propose data shapes for <source>"; or any former /wicked-garden:data:* invocation. Delegated deep passes: the `wicked-garden-data-engineer` worker skill.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 8.0 KB |
| refs/analysis-templates.md | markdown | 3.3 KB |
| refs/analyze.md | markdown | 5.1 KB |
| refs/data-examples.md | markdown | 4.9 KB |
| refs/data.md | markdown | 3.8 KB |
| refs/ml-templates.md | markdown | 4.0 KB |
| refs/ml.md | markdown | 5.7 KB |
| refs/pipeline-templates.md | markdown | 4.6 KB |
| refs/pipeline.md | markdown | 4.8 KB |
| refs/shapes.md | markdown | 6.1 KB |
refs/pipeline-templates.md
View on GitHub---
phase_relevance: ["design", "build"]
archetype_relevance: ["*"]
---
# Pipeline Output Templates
Standard templates for pipeline design and review documentation.
## Pipeline Design Document
```markdown
## Pipeline Design: {name}
**Author**: {name}
**Date**: {date}
**Status**: {draft/review/approved}
### Requirements
- **Source**: {system/database/API}
- **Target**: {destination system}
- **Frequency**: {hourly/daily/streaming}
- **Latency SLA**: {max delay from source}
- **Volume**: {GB/day or events/sec}
- **Retention**: {how long to keep data}
### Architecture
```
┌──────────┐ ┌───────────┐ ┌───────────┐ ┌──────────┐
│ Source │───►│ Extract │───►│ Transform │───►│ Target │
└──────────┘ └───────────┘ └───────────┘ └──────────┘
│ │
▼ ▼
┌───────────┐ ┌───────────┐
│ Staging │ │ Errors │
└───────────┘ └───────────┘
```
### Data Flow
#### 1. Extract
- **Method**: {full/incremental/CDC}
- **Source format**: {JSON/CSV/SQL}
- **Volume**: {rows/day}
- **Watermark**: {how to track progress}
#### 2. Transform
| Step | Operation | Input | Output |
|------|-----------|-------|--------|
| 1 | Parse | raw JSON | typed records |
| 2 | Validate | typed records | valid records + errors |
| 3 | Enrich | valid records | enriched records |
| 4 | Aggregate | enriched records | summary tables |
#### 3. Validate
- [ ] Schema validation
- [ ] Business rule validation
- [ ] Referential integrity
- [ ] Data quality checks
#### 4. Load
- **Pattern**: {append/merge/replace}
- **Partitioning**: {by date/region/etc}
- **Indexing**: {primary key, indexes}
### Error Handling
| Error Type | Detection | Response | Recovery |
|------------|-----------|----------|----------|
| Network | Connection timeout | Retry 3x | Alert if persists |
| Data format | Schema validation | Log + skip | Manual review |
| Business rule | Validation check | Dead letter queue | Reprocess |
| System | Exception | Alert + stop | Manual restart |
### Monitoring
| Metric | Target | Alert Threshold |
|--------|--------|-----------------|
| Row count | ±5% of avg | >10% deviation |
| Duration | <2 hours | >3 hours |
| Error rate | <0.1% | >1% |
| Freshness | <4 hours | >6 hours |
### Operations
#### Backfill Procedure
1. Set date range parameters
2. Run in staging mode first
3. Validate results
4. Run production load
#### Recovery Procedure
1. Check error logs
2. Fix root cause
3. Replay from last checkpoint
4. Verify data consistency
### Dependencies
| Dependency | Type | Owner | SLA |
|------------|------|-------|-----|
| {system} | upstream | {team} | {SLA} |
### Deployment
- **Orchestration**: {Airflow/Dagster/Prefect}
- **Schedule**: {cron expression}
- **Resources**: {CPU/memory/storage}
- **Environments**: {dev/staging/prod}
```
## Pipeline Review Report
```markdown
## Pipeline Review: {name}
**Reviewed**: {date}
**Reviewer**: {name}
### Summary
{2-3 sentence assessment of pipeline quality and readiness}
### Findings
#### Critical (Must Fix)
| # | Finding | Risk | Recommendation |
|---|---------|------|----------------|
| 1 | {issue} | Data loss | {action} |
#### High (Should Fix)
| # | Finding | Risk | Recommendation |
|---|---------|------|----------------|
| 1 | {issue} | Performance | {action} |
#### Medium (Consider)
| # | Finding | Risk | Recommendation |
|---|---------|------|----------------|
| 1 | {issue} | Maintenance | {action} |
### Code Quality Assessment
| Category | Score | Notes |
|----------|-------|-------|
| Structure | {1-10} | {comments} |
| Error Handling | {1-10} | {comments} |
| Testing | {1-10} | {comments} |
| Documentation | {1-10} | {comments} |
| Maintainability | {1-10} | {comments} |
**Overall**: {score}/50
### Performance Assessment
- **Current load**: {X rows in Y time}
- **Bottleneck**: {identified}
- **10x projection**: {will it scale?}
- **Recommendations**: {optimizations}
### Operational Readiness
- [ ] Monitoring configured
- [ ] Alerting configured
- [ ] Runbook documented
- [ ] Backfill tested
- [ ] Recovery tested
- [ ] On-call rotation assigned
### Verdict
**Status**: [PRODUCTION READY | NEEDS WORK | MAJOR REFACTOR]
**Blocking Issues**: {count}
**Required Actions Before Deploy**:
1. {action}
**Sign-off**: {name, date}
```
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |