Skip to content

topprismdata/cultivating-ml-agent / skills/examples/code-competition-artifact-pipeline

code-competition-artifact-pipeline

Verified

Use when: (1) Entering a Kaggle Code Competition that requires notebooks with no internet access, (2) Need to fork a public baseline that depends on external artifact datasets (pre-trained models, feature matrices, wheels), (3) Your fork fails with "module not found" or "file not found" errors, (4) Need to identify and attach the correct Kaggle datasets as notebook inputs. Key lessons: search kaggle datasets list for artifact bundles, check kernel-metadata.json dataset_sources field, the notebook's find_artifacts() function reveals expected paths. Validated on ROGII (7159→Pipeline A), Biohub (LB810 fork), NeuroGolf (7228 fork).

Code Competition Artifact Pipeline

Problem

Kaggle Code Competitions (no internet, notebook-only submission) often have community baselines that depend on external artifact datasets:

  • Pre-trained model weights (pickle/onnx)
  • Feature matrices (CSV/numpy)
  • Custom library wheels (.whl)

Forking these baselines fails unless you identify and attach ALL required datasets. The error messages are often cryptic ("file not found", "module not found").

Solution

Step 1: Read the Notebook's find_artifacts() Function

Most community baselines have a find_artifacts() or similar function that searches /kaggle/input/ for specific directory structures:

def find_artifacts():
    candidates = [
        "/kaggle/input/datasets/author/artifact-name",
        "/kaggle/input/artifact-name",
    ]
    # ... searches for repo/, weights/, wheels/, data/train.csv

This reveals: exact dataset names, expected file structure, and required Python packages.

Step 2: Search for Artifact Datasets

kaggle datasets list --search "competition-name artifacts"
kaggle datasets list --search "author-name"

Look for high-download datasets (500+ downloads = canonical bundle).

Step 3: Attach ALL Required Datasets

In kernel-metadata.json:

{
  "dataset_sources": [
    "author/competition-artifacts",
    "author/library-wheel"
  ]
}

Common missing datasets (from real failures):

CompetitionMissing DatasetError
ROGIIravaghi/wellbore-geology-prediction-artifacts"data/train.csv not found"
ROGIIphongnguyn23021656/koolbox-offline"No module named 'koolbox'"
Biohubthibautgoldsborough/cellmot-baseline-artifacts"repo/ not found"
NeuroGolf(embedded in notebook, no external needed)—

Step 4: Pipeline-Trim for Crashing Components

If the notebook has multiple pipelines (A/B) and one crashes:

  • Pipeline A succeeds but Pipeline B errors → kernel ERROR → no submission scored
  • Fix: delete Pipeline B cells, keep only Pipeline A + final submission write
  • Alternative: wrap Pipeline B in try/except

ROGII case study: Pipeline A (OOF RMSE 10.38) succeeded but Pipeline B (lik-PF) crashed. Trimming Pipeline B cells made the kernel COMPLETE, allowing the submission to be scored.

Step 5: CPU vs GPU Mode

  • CPU kernels: no GPU quota cost, but slower (inference-only mode for some notebooks)
  • GPU kernels: faster, but 30h/week quota limit shared across all competitions
  • Some notebooks REQUIRE GPU (e.g., Biohub UNet, ARC-AGI Duck harness)

Anti-Patterns

Anti-PatternFix
Fork without reading find_artifacts()Read the function to find dataset names
Attach only 1 dataset when 2+ neededCheck ALL import/glob patterns
Submit ERROR kernel expecting partial scoreCode competitions don't score ERRORs
Keep crashing Pipeline B cellsDelete them or wrap in try/except
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 321kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 197kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall