Skip to content

ericrisco/rsc-harness / skills/rag

Use when building grounded Q&A over your own corpus — chunk, retrieve hybrid, rerank, ground, cite chunk ids, refuse when the sources fall short — or when the right document is retrieved but the answer is still wrong, invented, or unmeasured. NOT operating the store itself — collection schema, HNSW ef_search, quantization (that is `vector-db`).

FileKindSize
SKILL.mdskill md11.4 KB
evals/README.mdmarkdown1.0 KB
evals/cases.yamlconfig3.6 KB
references/evaluation.mdmarkdown4.3 KB
references/pipeline.mdmarkdown6.1 KB
scripts/verify.shexecutablescript6.6 KB

evals/README.md

View on GitHub
# rag evals — how to run

These are routing and capability checks, not unit tests. Run the `should_trigger` prompts against
your skill router and confirm `rag` fires; run the `should_not_trigger` prompts and confirm the
router instead defers to the named sibling (`vector-db`, `embeddings-search`, `document-processing`,
`chatbot`, `structured-extraction`) — a miss there means the boundary in the description needs
sharpening. For the `capability` case, hand the scenario to the agent with this skill loaded and
grade the answer against the `must_include` rubric line by line (does it name all five pipeline
stages, hybrid + RRF, the rerank funnel, the grounding-plus-refusal contract, the eval gate, and
the two handoffs); each rubric item is pass/fail. You can do this by hand or wire it through the
`agent-eval` harness for a repeatable score. `scripts/verify.sh` is the static gate on the skill's
own artifacts (frontmatter, the grounding contract, the references, the RAGAS metric names) and is
separate from these capability checks.
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall