Skip to content

ericrisco/rsc-harness / skills/go

Use when writing, reviewing, testing, or shipping Go code and HTTP services: idioms, `%w` error wrapping, goroutine/context/errgroup concurrency, net/http 1.22 routing, log/slog, project layout, table-driven tests, Go hardening. NOT language-agnostic threat modeling (that is `security-audit`), NOT Dockerfile/CI shipping (that is `deployment`).

FileKindSize
SKILL.mdskill md15.2 KB
evals/README.mdmarkdown2.6 KB
evals/cases.yamlconfig5.1 KB
references/concurrency.mdmarkdown15.8 KB
references/http-services.mdmarkdown14.7 KB
references/testing.mdmarkdown10.3 KB
scripts/verify.shexecutablescript5.3 KB

evals/README.md

View on GitHub
# Eval harness — `go` skill

These evals are run by an **agent harness** (a Claude agent loading skills), not a pure
shell script. `cases.yaml` is the fixture; this file is the procedure. Two things are
measured: **triggering** (does the right skill fire?) and **capability** (does the skill
make the answer better?).

## 1. Triggering

Goal: the `go` skill fires on `should_trigger` prompts and stays silent on
`should_not_trigger` near-misses.

Setup:
- Load the **full skill catalog descriptions** (the routing layer sees every sibling:
  fastapi, nextjs, go, postgresdb, flutter, design, marketing, presentations,
  course-storytelling, building-agents, security-audit, deployment, harness, init).
- For each prompt, start a fresh agent session (no prior context bleed).

Procedure:
- Feed each `should_trigger.prompt`; record whether the agent invokes the `go` skill.
- Feed each `should_not_trigger.prompt`; record which skill (if any) it invokes, and
  confirm it matches the stated `route_to` (or `none`).
- Run **3–5 trials per prompt** (LLM routing is non-deterministic); a prompt passes if it
  fires correctly in the **majority** of trials.

Pass bar:
- **≥90% trigger accuracy** across all prompts (should_trigger fires AND should_not_trigger
  does not fire `go`).
- A `should_not_trigger` that routes to the *wrong* sibling but still avoids `go` is a
  partial pass — log it; routing precision matters for the catalog.

## 2. Capability

Goal: the skill **measurably improves** the answer, not just decorates it.

Procedure for each `capability.scenario`:
1. **Without** the skill: run the scenario with the skill body unavailable. Grade the
   answer against `must_include`.
2. **With** the skill: run the same scenario with `go/SKILL.md` (and references) loaded.
   Grade again against `must_include`.
3. Score = fraction of `must_include` points present and correct. Run 3 trials per
   condition and average.

Pass bar:
- **With** the skill: **≥80%** of `must_include` points covered.
- The with-skill score must **beat** the without-skill score by a clear margin (the skill
  is earning its place). If a baseline model already hits the rubric without the skill,
  the rubric is too easy — tighten it.

## Notes

- Grading is rubric-based and partly judgment (e.g. "no string-matching of error
  messages"); a human or a grader-agent reads each point against the output.
- Keep `cases.yaml` faithful to `SKILL.md`'s "When to use / When NOT to use" — when the
  skill's scope changes, update the cases in the same change.
- These evals do not run in CI as-is; they are a reproducible manual/agent procedure.
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall