Skip to content

ericrisco/rsc-harness / skills/data-cleaning

data-cleaning

Unverified

Use when a raw table is too dirty to trust — nulls, sentinels, duplicate rows, category sprawl, mixed types, bad dates — and you need a re-runnable clean() plus a schema gate that fails loud. NOT emitting .xlsx (that is spreadsheet-ops), NOT acquiring rows (that is data-scraper), NOT parsing PDF/HTML into rows (that is structured-extraction).

FileKindSize
SKILL.mdskill md14.3 KB
evals/README.mdmarkdown1.3 KB
evals/cases.yamlconfig4.0 KB
references/normalization-recipes.mdmarkdown5.4 KB
references/validation-patterns.mdmarkdown6.1 KB
scripts/verify.shexecutablescript4.0 KB

evals/cases.yaml

View on GitHub
skill: data-cleaning

should_trigger:
  - prompt: "Clean and dedupe this customer CSV and give me something I can re-run every month with the same result."
    why: Core ask — reproducible cleaning pipeline plus dedupe; the headline use case.
  - prompt: "My integer ID column came in as 1001.0 floats and a bunch of rows are duplicated. Fix it."
    why: Non-obvious dtype footgun — only this skill explains read-typing (dtype_backend) before deterministic dedupe.
  - prompt: "Normalize these country values — USA, U.S., United States all mean the same thing."
    why: Category normalization via a mapping table; the unmapped→quarantine pattern is this skill's.
  - prompt: "Add a validation gate so bad rows fail loudly before they ever reach my model."
    why: Schema-contract / pandera framing — declare types, ranges, allowed values; fail loud.
  - prompt: "Quarantine the rows that don't pass instead of silently dropping them."
    why: Non-obvious quarantine pattern unique to this skill; not a generic cleaning ask.
  - prompt: "Limpiar y deduplicar este dataset y dejar un pipeline repetible."
    why: Spanish trigger — same reproducible clean+dedupe intent, no English keyword present.
  - prompt: "Why do my dates parse wrong sometimes and a few rows look duplicated after import?"
    why: Symptom-led, never says 'clean' — ambiguous date format + dedupe both live here.

should_not_trigger:
  - prompt: "Export this cleaned DataFrame to Excel with a summary chart and conditional formatting."
    route_to: spreadsheet-ops
    why: Output is a styled sheet/formulas/chart; cleaning already ended at the clean table.
  - prompt: "Scrape all the product prices off this catalog website."
    route_to: data-scraper
    why: Acquisition — the rows aren't held yet; cleaning only starts once you have tabular rows.
  - prompt: "Pull the line items out of these invoice PDFs into a table."
    route_to: structured-extraction
    why: Parsing fields out of unstructured blobs into rows, not cleaning rows you already hold.
  - prompt: "Aggregate this 5GB CSV with SQL fast on my laptop."
    route_to: duckdb
    why: OLAP speed over multi-GB files is the bottleneck, not correctness/normalization.
  - prompt: "Build the KPI dashboard from this already-clean table."
    route_to: business-intelligence
    why: Consumes the clean output; the boundary is the validated table.
  - prompt: "Forecast next quarter's revenue from this monthly clean series."
    route_to: forecasting
    why: Modeling on top of clean data, not producing the clean data.

capability:
  - scenario: >
      Given a messy customers CSV — an id column read as floats with .0, "N/A" and 999 sentinels in age,
      duplicate rows per customer, country values USA / U.S. / United States, and malformed dates — produce
      a reproducible clean() pipeline plus a pandera schema that types, normalizes, dedupes deterministically,
      validates, and quarantines failures rather than dropping them.
    must_include:
      - Reads with explicit dtype control (dtype_backend="pyarrow" or numpy_nullable + na_values) so ids stay int, not float
      - Profiles first (nulls %, dup count, per-column dtype, cardinality) to drive the plan
      - Normalizes strings with strip/casefold/NFKC before dedupe
      - Maps categories with a mapping table (dict), unmapped → NA/quarantine, not a regex pile
      - Turns sentinels (-1/999/"N/A") into NA, with an explicit clip-vs-reject range decision
      - Parses dates with an explicit format and errors="coerce", then checks the coerced-NaT count
      - Deduplicates deterministically — explicit subset key + stable sort_values + keep
      - Handles missing per column; no bare fillna(0); impute adds a _was_missing flag
      - Defines a pandera DataFrameModel with coerce, nullable, Field checks (isin/ge/le/unique), and lazy=True
      - Quarantines failing rows to an artifact instead of silently dropping them
      - Writes to a NEW output artifact, leaving the raw source untouched
      - Logs a row-count in/out/coerced/quarantined diff and pins library versions
canvas-designanthropics/skillsCreate beautiful visual art in .png and .pdf documents using design philosophy. You should use this skill when the user asks to create a poster, piece of art, design, or other static piece. Create original visual designs, never copying existing artists' work to avoid copyright violations.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
docxanthropics/skillsUse this skill whenever the user wants to create, read, edit, or manipulate Word documents (.docx files) or Word templates (.dotx files). Triggers include: any mention of 'Word doc', 'word document', '.docx', '.dotx', or requests to produce professional documents with formatting like tables of contents, headings, page numbers, or letterheads. Also use when extracting or reorganizing content from .docx or .dotx files, inserting or replacing images in documents, performing find-and-replace in Word files, working with tracked changes or comments, or converting content into a polished Word document. If the user asks for a 'report', 'memo', 'letter', 'template', or similar deliverable as a Word or .docx file, use this skill. Do NOT use for PDFs, spreadsheets, Google Docs, or general coding tasks unrelated to document generation.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
pdfanthropics/skillsUse this skill whenever the user wants to do anything with PDF files. This includes reading or extracting text/tables from PDFs, combining or merging multiple PDFs into one, splitting PDFs apart, rotating pages, adding watermarks, creating new PDFs, filling PDF forms, encrypting/decrypting PDFs, extracting images, and OCR on scanned PDFs to make them searchable. If the user mentions a .pdf file or asks to produce one, use this skill.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
pptxanthropics/skillsUse this skill any time a .pptx or .potx file is involved in any way — as input, output, or both. This includes: creating slide decks, pitch decks, or presentations; reading, parsing, or extracting text from any .pptx or .potx file (even if the extracted content will be used elsewhere, like in an email or summary); editing, modifying, or updating existing presentations; combining or splitting slide files; working with templates (.potx), layouts, speaker notes, or comments. Trigger whenever the user mentions "deck," "slides," "presentation," or references a .pptx or .potx filename, regardless of what they plan to do with the content afterward. If a .pptx or .potx file needs to be opened, created, or touched, use this skill.Stars: 198kUpdated Claude CodeCodexVerifiedInstall