aiopshwang/data-analysis-ml-agent-skills / skills/shipping-reproducible-results
shipping-reproducible-results
VerifiedPackage completed data analysis and ML work so an independent recipient can reproduce the claimed results, verify artifact lineage, and operate the handoff within its stated scope. Use when finalizing a project, study, model package, or review bundle; not for deploying to a live system.
Shipping Reproducible Results
Ship an evidence-bearing result, not merely code that once ran. The package must connect the approved question to immutable inputs, an executable workflow, generated outputs, validation evidence, and an explicit handoff boundary.
Freeze the result contract
Before packaging, record:
- the approved question, population, unit of analysis, time window, and exclusions;
- the exact claims and acceptance criteria being delivered;
- which outputs are authoritative and which are exploratory;
- the source snapshot or retrieval contract, including access limitations;
- the intended execution environment and supported operating scope.
Do not silently expand a validated result from a sample to a population, from retrospective analysis to prospective use, or from one environment to another.
Build the provenance chain
Make each authoritative output traceable through:
source identity -> immutable snapshot or query -> transformation -> configuration
-> code revision -> environment -> model artifact -> evaluation -> published output
Record content hashes for immutable local artifacts and durable identifiers for remote sources. Preserve raw inputs when permitted; otherwise preserve the exact query, retrieval time, schema, row counts, access requirements, and a safe fixture sufficient to test the pipeline.
Separate source data, intermediate artifacts, final outputs, caches, and human edits. Never require a recipient to guess which file is canonical.
Make execution deterministic enough to audit
Provide one documented entry point from a clean state. Pin direct dependencies and runtime versions; record relevant system libraries and hardware. Seed every controllable source of randomness and document remaining nondeterminism rather than promising bitwise equality when the stack cannot provide it.
Packaging does not authorize restricted-data access, material compute, credential use, external transfer, or publication. Verify the relevant authority before those actions. When execution is not authorized or available, prepare the reproducible contract and label the unexecuted stages instead of running them.
Define acceptable reproduction precisely:
- exact hashes for deterministic artifacts;
- numeric tolerances for floating-point outputs;
- statistical or ranking tolerances for nondeterministic training;
- invariant schemas, row counts, and decision outcomes where values may vary.
Fail loudly on missing inputs, schema drift, stale caches, or incompatible artifacts.
Prove the full path
Run from the documented clean entry point without relying on notebook state, shell history, undeclared credentials, or files outside the package. Verify every material claim with evidence of matching scope.
A green unit test is not proof of complete reproduction. The completion matrix must distinguish:
- environment creation;
- input acquisition or verification;
- preprocessing and feature generation;
- model loading or training;
- evaluation and independent validation;
- report or artifact generation;
- comparison with the authoritative expected outputs.
If a full rerun is impossible because data, compute, or credentials are unavailable, ship the maximum verifiable subset and mark the unexecuted stages as unproven. Do not call the package fully reproducible.
Handoff to a named owner
Document how to run, verify, interpret, update, and safely retire the result. Include known limitations, prohibited uses, external dependencies, expected costs, credential ownership, failure signals, and escalation contacts. Preserve rejected hypotheses and failed experiments when they affect future maintenance or interpretation.
Use assets/reproduction-handoff.md as the release document and assets/run-manifest.yaml as a machine-readable starting point. For verification levels, lineage requirements, and clean-room acceptance rules, read references/reproduction-contract.md.
Release gate
Release only when the package contains no secrets, unintended or unauthorized private data, machine-specific absolute paths, undeclared dependencies, or ambiguous canonical artifacts. Keep approved restricted artifacts in their authorized secure location and publish only permitted references or sanitized fixtures. External sharing and public release require separate authority. The final status must be one of:
- reproduced: an independent clean run met the declared acceptance criteria;
- ready for independent reproduction: the complete package and entry point are prepared, but an independent clean run has not yet been performed;
- replayable with constraints: the executable path is present but named stages require unavailable access or resources;
- archival evidence only: outputs and provenance are preserved, but the computation cannot currently be rerun.
State the status prominently. Never upgrade it based on intent or partial checks.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |