ericrisco/rsc-harness / skills/course-storytelling
course-storytelling
VerifiedUse when lesson or course content is correct but forgettable and a concept has to LAND: profiles the learner, breaks the blocking false belief, then rebuilds it as epiphany story → named model → grounded analogy → proof → so-what. NOT outcomes, assessment or module order (that is `course-builder`), NOT slide visuals (that is `presentations`).
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 14.2 KB |
| evals/README.md | markdown | 2.7 KB |
| evals/cases.yaml | config | 5.7 KB |
| references/brunson-frameworks.md | markdown | 11.1 KB |
| references/concept-landing-recipe.md | markdown | 7.3 KB |
| references/course-analysis.md | markdown | 6.9 KB |
| references/learner-grounding.md | markdown | 9.2 KB |
| references/mental-models.md | markdown | 6.8 KB |
| scripts/verify.shexecutable | script | 9.4 KB |
evals/README.md
View on GitHub# Eval harness — `course-storytelling`
This is an **agent-run** eval, not a shell script. You drive a Claude Code agent
(or equivalent) and judge its behaviour against `cases.yaml`. There is no
automated grader here; a human or a judge-agent reads the transcript and scores it.
`cases.yaml` has three blocks: `should_trigger` (6), `should_not_trigger` (5),
and `capability` (2 scenarios with rubrics).
## A. Triggering accuracy
Goal: the skill fires on real teaching-narrative work and stays quiet on near-misses.
1. Load **only** `course-storytelling` into the agent (no sibling skills loaded,
so a miss can't be masked by another skill picking up the slack).
2. For **each** prompt in `should_trigger` and `should_not_trigger`, start a fresh
session and paste the prompt verbatim. Run **3–5 trials** per prompt (the
trigger decision is stochastic).
3. Record per trial:
- `should_trigger` → PASS if the agent invokes/announces `course-storytelling`.
- `should_not_trigger` → PASS if it does **not** invoke it. Bonus: it routes to
the `route_to` sibling named in the case (or correctly declines when `none`).
4. **Pass bar: ≥90% correct decisions** across all trials (both blocks combined).
Any prompt that fails on a majority of its trials is a real defect — fix the
SKILL.md description/trigger list, don't loosen the case.
## B. Capability uplift (with vs without)
Goal: the skill **measurably improves** the teaching output, not just fires.
1. For each `capability` scenario, run it **twice**:
- **WITHOUT** the skill (baseline — agent answers from general knowledge).
- **WITH** `course-storytelling` loaded.
2. Score each output against that scenario's `must_include` rubric: fraction of
checkable points actually present.
3. **Pass bar:**
- WITH the skill: **≥80% of rubric points covered.**
- The WITH score must **clearly beat** WITHOUT (expect the baseline to skip the
learner-grounding gate, invent proof, state the insight instead of building an
epiphany, and end on a summary — all rubric misses).
- Scenario 2 specifically checks the **hard STOP**: without the skill the agent
will usually just answer; with it, it must refuse-and-interview first.
## Honesty notes
- Trials are stochastic — report the actual trial counts and pass rates, don't
round a 2/5 up to "passes".
- A `should_not_trigger` that fires is as much a defect as a `should_trigger` that
misses; both go in the report.
- If a case is wrong (ambiguous prompt, sibling overlap is genuinely 50/50), fix
`cases.yaml` and say so — don't quietly grade around it.
- Capability grading is judgement, not grep. The `scripts/verify.sh` in the skill
only covers the greppable subset (jargon/story/name flags); the rubric here is
the real bar.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| canvas-designanthropics/skillsCreate beautiful visual art in .png and .pdf documents using design philosophy. You should use this skill when the user asks to create a poster, piece of art, design, or other static piece. Create original visual designs, never copying existing artists' work to avoid copyright violations. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| docxanthropics/skillsUse this skill whenever the user wants to create, read, edit, or manipulate Word documents (.docx files) or Word templates (.dotx files). Triggers include: any mention of 'Word doc', 'word document', '.docx', '.dotx', or requests to produce professional documents with formatting like tables of contents, headings, page numbers, or letterheads. Also use when extracting or reorganizing content from .docx or .dotx files, inserting or replacing images in documents, performing find-and-replace in Word files, working with tracked changes or comments, or converting content into a polished Word document. If the user asks for a 'report', 'memo', 'letter', 'template', or similar deliverable as a Word or .docx file, use this skill. Do NOT use for PDFs, spreadsheets, Google Docs, or general coding tasks unrelated to document generation. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| pdfanthropics/skillsUse this skill whenever the user wants to do anything with PDF files. This includes reading or extracting text/tables from PDFs, combining or merging multiple PDFs into one, splitting PDFs apart, rotating pages, adding watermarks, creating new PDFs, filling PDF forms, encrypting/decrypting PDFs, extracting images, and OCR on scanned PDFs to make them searchable. If the user mentions a .pdf file or asks to produce one, use this skill. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| pptxanthropics/skillsUse this skill any time a .pptx or .potx file is involved in any way — as input, output, or both. This includes: creating slide decks, pitch decks, or presentations; reading, parsing, or extracting text from any .pptx or .potx file (even if the extracted content will be used elsewhere, like in an email or summary); editing, modifying, or updating existing presentations; combining or splitting slide files; working with templates (.potx), layouts, speaker notes, or comments. Trigger whenever the user mentions "deck," "slides," "presentation," or references a .pptx or .potx filename, regardless of what they plan to do with the content afterward. If a .pptx or .potx file needs to be opened, created, or touched, use this skill. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |