ericrisco/rsc-harness / skills/python
python
VerifiedUse when the task is Python itself, in any framework or none: PEP 695 generics, mypy --strict typing, dataclass/Protocol/TypedDict/Enum choices, asyncio.TaskGroup, stdlib idioms, src/ layout + pyproject.toml with uv, ruff+mypy+pytest gate. NOT a FastAPI/ASGI service (that is fastapi), NOT a deep pytest suite (that is testing-py).
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 14.9 KB |
| evals/README.md | markdown | 1.4 KB |
| evals/cases.yaml | config | 3.5 KB |
| references/async.md | markdown | 5.3 KB |
| references/stdlib.md | markdown | 5.1 KB |
| references/typing.md | markdown | 5.8 KB |
| scripts/verify.shexecutable | script | 4.1 KB |
evals/README.md
View on GitHub# Eval harness — `python` skill
These cases are read by an **agent harness** (a Claude agent with the skill catalog loaded),
not a plain shell script. `cases.yaml` is the fixture; this file is the procedure. Two things
are measured: **triggering** (does `python` fire on language-level prompts and stay silent on
near-misses that belong to `fastapi`, `testing-py`, `django`, `security-audit`, `deployment`?)
and **capability** (does loading the skill make a scaffold answer measurably better?).
**Triggering:** load the full sibling descriptions, start a fresh session per prompt, feed each
`should_trigger.prompt` and confirm `python` fires; feed each `should_not_trigger.prompt` and
confirm it routes to the stated `route_to` and does *not* fire `python`. LLM routing is
non-deterministic — run 3–5 trials each and pass on the majority; target ≥90% accuracy.
**Capability:** run the `capability.scenario` twice — once with the skill body unavailable, once
with `python/SKILL.md` (and references) loaded — and grade each output against the `must_include`
rubric (fraction of points present and correct, averaged over 3 trials). Pass bar: ≥80% with the
skill, and a clear margin over the without-skill baseline. No network is required; grading is
rubric-based and partly judgment (e.g. "no mutable default argument"). Keep `cases.yaml` faithful
to SKILL.md's "When to use / When NOT to use" — change them together.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |