Skip to content

mikeparcewski/wicked-garden / skills/data

wicked-garden-data

Verified

Data engineering, analysis, and ML toolkit. One skill, routed sub-actions: analyze (interactive EDA on a CSV/Excel/data file), profile / validate / quality (schema-level data engineering ops), ml review / ml pipeline (model review + training-pipeline design), pipeline design / pipeline review (ETL architecture), ontology (recommend Schema.org, Dublin Core, DCAT, FOAF, GoodRelations, SKOS, or a custom shape for a dataset), shapes (a live source's data shapes, proposed for review). Use when: "analyze this CSV/Excel/data file" / "explore this dataset" / one-off data exploration; "profile this dataset" / "validate data against a schema" / "generate a data quality report"; "review this ML model" / "design an ML training pipeline"; "design a data pipeline" / "review this ETL"; "recommend an ontology for this data" / "map columns to a public ontology"; "propose data shapes for <source>"; or any former /wicked-garden:data:* invocation. Delegated deep passes: the `wicked-garden-data-engineer` worker skill.

FileKindSize
SKILL.mdskill md8.0 KB
refs/analysis-templates.mdmarkdown3.3 KB
refs/analyze.mdmarkdown5.1 KB
refs/data-examples.mdmarkdown4.9 KB
refs/data.mdmarkdown3.8 KB
refs/ml-templates.mdmarkdown4.0 KB
refs/ml.mdmarkdown5.7 KB
refs/pipeline-templates.mdmarkdown4.6 KB
refs/pipeline.mdmarkdown4.8 KB
refs/shapes.mdmarkdown6.1 KB

refs/data-examples.md

View on GitHub
---
phase_relevance: ["design", "build"]
archetype_relevance: ["*"]
---
# Data Engineering Examples

Detailed examples for data profiling, schema validation, and quality assessment.

## Profiling Output Example

```json
{
  "file": "sales.csv",
  "rows": 10000,
  "columns": 8,
  "schema": [
    {
      "name": "order_id",
      "type": "string",
      "null_rate": 0.0,
      "cardinality": 10000,
      "sample": ["ORD001", "ORD002", "ORD003"]
    },
    {
      "name": "amount",
      "type": "decimal",
      "null_rate": 0.02,
      "stats": {
        "min": 10.50,
        "max": 9999.99,
        "mean": 523.45,
        "median": 299.00
      }
    },
    {
      "name": "customer_id",
      "type": "integer",
      "null_rate": 0.0,
      "cardinality": 5000
    },
    {
      "name": "order_date",
      "type": "date",
      "null_rate": 0.0,
      "min": "2025-01-01",
      "max": "2025-12-31"
    }
  ],
  "quality_score": 95,
  "issues": [
    "Column 'email' has 2% null values"
  ]
}
```

## Schema Definition Example

```json
{
  "columns": [
    {
      "name": "user_id",
      "type": "integer",
      "nullable": false,
      "constraints": {
        "unique": true,
        "min": 1
      }
    },
    {
      "name": "email",
      "type": "string",
      "nullable": false,
      "constraints": {
        "pattern": "^[\\w\\.-]+@[\\w\\.-]+\\.\\w+$"
      }
    },
    {
      "name": "created_at",
      "type": "datetime",
      "nullable": false
    },
    {
      "name": "status",
      "type": "string",
      "nullable": false,
      "constraints": {
        "enum": ["active", "inactive", "pending"]
      }
    },
    {
      "name": "balance",
      "type": "decimal",
      "nullable": true,
      "constraints": {
        "min": 0,
        "precision": 2
      }
    }
  ]
}
```

## Quality Report Example

```markdown
## Data Quality Report

**Dataset**: sales.csv
**Date**: 2026-01-24
**Rows**: 10,000

### Overall Score: 87/100

### Dimension Scores
| Dimension | Score | Status |
|-----------|-------|--------|
| Completeness | 95 | PASS |
| Uniqueness | 98 | PASS |
| Validity | 85 | WARNING |
| Consistency | 70 | FAIL |

### Critical Issues
1. **Validity**: 15% of 'phone' values don't match pattern
2. **Consistency**: 30% of records have order_date > ship_date

### Issue Details

#### Issue 1: Invalid Phone Format
- **Column**: phone
- **Pattern**: Expected `^\\+?[0-9]{10,14}$`
- **Violations**: 1,500 rows (15%)
- **Samples**: "123-456", "N/A", "(555)123-4567"
- **Root Cause**: No input validation at data entry

#### Issue 2: Date Consistency
- **Rule**: ship_date >= order_date
- **Violations**: 3,000 rows (30%)
- **Impact**: Analytics reports show negative fulfillment times
- **Root Cause**: Timezone mismatch between systems

### Recommendations
1. Add validation for phone format at source
2. Implement business rule: ship_date >= order_date
3. Standardize timezone handling

**Action Required**: Fix consistency issues before using for analytics
```

## Common Workflows

All invocations below are sub-actions of the `wicked-garden-data` skill.

### New Dataset Analysis

1. **Profile** to understand structure
```
profile new_data.csv
```

2. **Validate** against expected schema (if available)
```
validate --schema expected.json --data new_data.csv
```

3. **Generate quality report**
```
quality new_data.csv
```

4. **Document issues** as a task — subject `Data Quality: Fix phone format`, metadata `event_type=task, chain_id=data-quality.root, source_agent=data-profiler, priority=P1` — in the harness's task list where it has one (the `wicked-garden-workflow` skill's `refs/integration.md` carries the field list and the harness-specific Hand-off), else your working notes

5. **Recommend remediation** with specific actions

### Schema Migration

1. Profile source data
2. Design target schema based on profile
3. Validate sample data against target
4. Identify transformation requirements:
   - Type conversions
   - Null handling
   - Constraint additions
5. Document migration plan

### Quality Monitoring

1. Profile dataset regularly (daily/weekly)
2. Compare metrics over time
3. Alert on quality degradation:
   - Null rate increase >5%
   - New duplicate patterns
   - Type conformance drops
4. Track improvement initiatives via native TaskCreate

## SQL Quality Queries

```sql
-- Completeness check
SELECT
  column_name,
  COUNT(*) as total,
  SUM(CASE WHEN column_name IS NULL THEN 1 ELSE 0 END) as nulls,
  ROUND(100.0 * SUM(CASE WHEN column_name IS NULL THEN 1 ELSE 0 END) / COUNT(*), 2) as null_pct
FROM data
GROUP BY ALL;

-- Uniqueness check
SELECT
  column_name,
  COUNT(*) as total,
  COUNT(DISTINCT column_name) as distinct_count,
  COUNT(*) - COUNT(DISTINCT column_name) as duplicates
FROM data
GROUP BY ALL;

-- Validity check (pattern matching)
SELECT
  COUNT(*) FILTER (WHERE email NOT LIKE '%@%.%') as invalid_emails
FROM data;

-- Consistency check (cross-field rules)
SELECT
  COUNT(*) FILTER (WHERE ship_date < order_date) as date_violations
FROM data;
```
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall