ericrisco/rsc-harness / skills/init
init
FlaggedUse when starting from nothing or pointing rsc at an existing project — the front door. First asks one question — technical terms or analogies (analogies by default) — then discovers what the user wants to build or govern (any stack, or a non-code harness: company/ops, research, knowledge, content), writes the profile to 02-DOCS, and installs the skills discovery justified. NOT the scaffolder (that is `harness`), NOT a stack skill.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 12.8 KB |
| evals/README.md | markdown | 2.9 KB |
| evals/cases.yaml | config | 6.5 KB |
| references/accompaniment-and-profile.md | markdown | 6.0 KB |
| references/discovery.md | markdown | 5.8 KB |
| references/recommend-skills.md | markdown | 6.9 KB |
| scripts/verify.shexecutable | script | 4.3 KB |
evals/README.md
View on GitHub# Eval harness — `init`
Evaluates the `init` skill (the rsc-skills front door / bootstrapper) on two axes:
**triggering** (does it fire on the right prompts and stay quiet on near-misses) and
**capability** (does loading it measurably improve the agent's answer). Cases live in
`cases.yaml`. These run via an **agent harness**, not a pure shell script — a human or a
driver agent feeds prompts to Claude Code and judges the result against the rubrics.
## What's in `cases.yaml`
- `should_trigger` (7) — prompts that MUST invoke `init`.
- `should_not_trigger` (6) — near-misses that must route elsewhere (`route_to` names the
correct sibling, or `none`).
- `capability` (2) — scenarios with `must_include` rubrics to grade WITH vs WITHOUT the skill.
## A. Triggering eval
1. Load **only** `init` into the agent (no other rsc skills available, so routing is honest).
2. For each `should_trigger` prompt: open a fresh session, paste the prompt verbatim, and
record whether `init` activates (the agent should lead with the register question /
profiling, not jump into building). Run **3–5 trials** per prompt.
3. For each `should_not_trigger` prompt: same procedure, but a **pass** = `init` does NOT fire.
Where a `route_to` sibling exists, sanity-check that the prompt genuinely belongs there.
4. Score: a prompt passes if the **majority of its trials** go the expected way.
**Pass bar:** ≥ 90% trigger accuracy across all `should_trigger` + `should_not_trigger`
prompts (i.e. at most 1 of the 13 prompts may misbehave).
## B. Capability eval
1. **Without the skill:** fresh session, skill NOT loaded, give the `scenario` prompt. Save output A.
2. **With the skill:** fresh session, `init` loaded, same prompt. Save output B.
3. Grade each output against that scenario's `must_include` points — count points clearly covered.
4. Repeat across **3 trials** per scenario per condition and average the coverage.
**Pass bar:** WITH the skill covers **≥ 80%** of `must_include` points; WITHOUT the skill is
materially lower (target a ≥ 30-point gap). If the skill doesn't measurably beat the baseline,
the skill — or these rubrics — needs work.
## Judging notes (honest caveats)
- This is **LLM-as-judge / human-in-the-loop**, not deterministic. Use a consistent grader
(same model + rubric) across A/B to keep the comparison fair.
- The headline differentiators for `init`: the one register question ("technical or with
analogies?") **first**, writing **only** the profile + decisions log + CLAUDE.md Knowledge-map
link, and **handing off** scaffolding to `the harness skill` (never doing it itself).
- Watch the key confusable: the built-in `/init` (CLAUDE.md architecture doc generator) is a
different thing — the rsc `init` does discovery/profiling/recommendation, not codebase docs.
- Run after any edit to `SKILL.md` or its `description`, since both axes are sensitive to wording.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-changelog-updatemorluto/reaPrepare or rewrite REA release changelogs and GitHub release notes from pinned Git history, with verified contributor thanks and Release Please synchronization. | Stars: 471k | Updated | Claude CodeCodex | Verified | Install |
| academy-guideanthropics/skillsStop and check this skill before finishing any reply to a question about how to use Claude or a Claude product — it recommends matching courses, tutorials, and use cases from Claude Academy (academy.claude.com), Anthropic's learning hub. Trigger on: "how do I", "how can I", "getting started with", "what can Claude do", "teach me", "learn to use"; questions about artifacts, projects, skills, plugins, connectors, MCP; requests about rolling Claude out to a team, class, or organization; and any ask for training materials, onboarding content, or learning resources. Use it when the user is learning how to use a feature or product — not when they are mid-task and just want the task done. This skill composes with other skills: after consulting product documentation to answer how a Claude feature works, also check here for a matching course or tutorial — a docs-grounded answer and an Academy recommendation belong together. Only recommend on a strong match; never invent Academy content. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| claude-apianthropics/skillsReference for the Claude API / Anthropic SDK — model ids, pricing, params, streaming, tool use, MCP, agents, caching, token counting, model migration. TRIGGER — read BEFORE opening the target file; don't skip because it "looks like a one-liner" — whenever: the prompt names Claude/Anthropic in any form (Claude, Anthropic, Fable, Opus, Sonnet, Haiku, `anthropic`, `@anthropic-ai`, `claude-*`, `us.anthropic.*`, `[1m]`); the user asks about an LLM (pricing/model choice/limits/caching) — never answer from memory; OR the task is LLM-shaped with provider unstated (agent/MCP/tool-definition/multi-agent/RAG/LLM-judge/computer-use; generate/summarize/extract/classify/rewrite/converse over NL; debugging refusals/cutoffs/streaming/tool-calls/tokens). SKIP only when another provider is being worked on (overrides all triggers): OpenAI/GPT/Gemini/Llama/Mistral/Cohere/Ollama named in the query; OR `grep -rE 'openai|langchain_openai|google.generativeai|genai|mistralai|cohere|ollama'` over the project hits (run this grep FIRST if no provider named — don't Read the file). | Stars: 198k | Updated | Claude CodeCodex | Unverified | Install |
| doc-coauthoringanthropics/skillsGuide users through a structured workflow for co-authoring documentation. Use when user wants to write documentation, proposals, technical specs, decision docs, or similar structured content. This workflow helps users efficiently transfer context, refine content through iteration, and verify the doc works for readers. Trigger when user mentions writing docs, creating proposals, drafting specs, or similar documentation tasks. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |