ericrisco/rsc-harness / skills/modal
modal
VerifiedUse when running Python or GPU workloads serverlessly on Modal — modal.App, inline container Images, gpu= on @app.function, Volumes for weight caching, Cron schedules, ASGI endpoints, modal run vs serve vs deploy. NOT managed prediction APIs with no container of your own (that is replicate); NOT SSH-able GPU boxes rented by the hour (that is runpod).
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 13.6 KB |
| evals/README.md | markdown | 1.8 KB |
| evals/cases.yaml | config | 4.1 KB |
| references/images-gpu-cookbook.md | markdown | 5.0 KB |
| references/web-and-scaling.md | markdown | 4.7 KB |
| scripts/verify.shexecutable | script | 4.6 KB |
evals/README.md
View on GitHub# Eval harness — `modal` skill
These cases are run by an **agent harness** (an LLM agent with the skill catalog available for
routing), not a plain shell script. `cases.yaml` is the fixture; this file is the procedure.
No Modal account or credentials are needed — every check is about *routing* and the *generated
code rubric*, not live execution against Modal.
## 1. Triggering accuracy
Load only the skill descriptions (this `modal` one plus the bare names of routing siblings:
replicate, runpod, fastapi, docker, python, llm-pipeline), not the bodies. For each prompt in
`should_trigger` / `should_not_trigger`, ask which skill the agent would invoke, running 3–5
trials per prompt. `should_trigger` passes if `modal` wins the majority; `should_not_trigger`
passes if `modal` does **not** fire (bonus if it routes to the named `route_to` sibling). Pass
bar: ≥90% of prompts, and no near-miss may fire `modal` in a majority of its trials. The
Catalan/Spanish and the `gpu=modal.gpu.A100()` prompts are the ones to watch for flakiness.
## 2. Capability uplift (with vs without)
For each `capability` scenario, run the agent twice — once with no `modal` skill loaded, once
with it fully loaded — and grade each output against the scenario's `must_include` checklist
(a point counts only if genuinely correct, not merely mentioned). Pass bar: WITH-skill
coverage ≥80% of the rubric, and WITH must clearly beat WITHOUT. The load-bearing points are
the string GPU form, `Volume.from_name(create_if_missing=True)` with commit/reload, correct
web-decorator stack order under `@app.function`, and `modal deploy` (not `run`) for anything
persistent. Any rubric point the skill never produces is a gap to fix in `SKILL.md`.
This is judgment-based, not byte-exact; record trial counts and the judge (human or model)
alongside results for reproducibility.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |