ericrisco/rsc-harness / skills/rag
rag
VerifiedUse when building grounded Q&A over your own corpus — chunk, retrieve hybrid, rerank, ground, cite chunk ids, refuse when the sources fall short — or when the right document is retrieved but the answer is still wrong, invented, or unmeasured. NOT operating the store itself — collection schema, HNSW ef_search, quantization (that is `vector-db`).
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 11.4 KB |
| evals/README.md | markdown | 1.0 KB |
| evals/cases.yaml | config | 3.6 KB |
| references/evaluation.md | markdown | 4.3 KB |
| references/pipeline.md | markdown | 6.1 KB |
| scripts/verify.shexecutable | script | 6.6 KB |
references/pipeline.md
View on GitHub# references/pipeline.md — the reference RAG pipeline
This is the long-form companion to the SKILL body: the full contextual-retrieval prompt, the
hybrid + RRF fusion, the rerank funnel parameters, the complete grounding system prompt, and a
minimal runnable retrieve → rerank → answer skeleton. Provider-neutral with concrete Cohere /
Anthropic examples. Adapt names; keep the contracts.
## 1. Contextual retrieval (full)
Anthropic's technique: before embedding and before BM25 indexing, prepend a 1–2 sentence
LLM-written blurb that situates each chunk inside its document. Reductions in failed retrieval:
~35% (contextual embeddings), ~49% (+ contextual BM25), ~67% (+ rerank)
(anthropic.com/news/contextual-retrieval, 2024-09; accessed 2026-06-02).
Cache the whole-document portion of the prompt so you pay for it once per document, not once per
chunk — this is what makes the technique affordable at scale.
```text
<document>
{{WHOLE_DOCUMENT}}
</document>
Here is the chunk we want to situate within the whole document:
<chunk>
{{CHUNK_TEXT}}
</chunk>
Give a short, standalone context (1–2 sentences) that situates this chunk within the overall
document, to improve search retrieval of the chunk. Answer ONLY with the succinct context and
nothing else.
```
```python
# Generate context per chunk, then index context + chunk for BOTH dense and BM25.
def contextualize(client, doc_text, chunk_text):
msg = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=120,
system=[{"type": "text",
"text": f"<document>\n{doc_text}\n</document>",
"cache_control": {"type": "ephemeral"}}], # cache the doc
messages=[{"role": "user", "content":
f"Here is the chunk:\n<chunk>{chunk_text}</chunk>\n"
"Give a 1-2 sentence standalone context for retrieval. Context only."}],
)
return msg.content[0].text.strip()
def indexable(doc_text, chunk):
ctx = contextualize(client, doc_text, chunk["text"])
chunk["context"] = ctx
chunk["embed_input"] = f"{ctx}\n{chunk['text']}" # embed this
chunk["bm25_input"] = f"{ctx} {chunk['text']}" # index this for keyword
return chunk
```
## 2. Hybrid retrieval + Reciprocal Rank Fusion
Dense search catches paraphrase; BM25 catches exact tokens. Run both and fuse by rank, not by
raw score (scores from two systems are not comparable). RRF score for a document is the sum over
result lists of `1 / (k + rank)`, with `k ≈ 60` a standard constant.
```python
def rrf(*ranked_lists, k=60, top_n=150):
scores = {}
seen = {}
for lst in ranked_lists: # each: list of chunks in rank order
for rank, chunk in enumerate(lst):
cid = chunk["chunk_id"]
scores[cid] = scores.get(cid, 0.0) + 1.0 / (k + rank)
seen[cid] = chunk
fused = sorted(seen.values(), key=lambda c: scores[c["chunk_id"]], reverse=True)
return fused[:top_n]
dense = vector_store.search(query_vec, limit=150) # ../vector-db owns this call
keyword = bm25.search(query, limit=150)
candidates = rrf(dense, keyword, top_n=150)
```
The actual store-side hybrid (named vectors, sparse-dense, server-side fusion in Qdrant/Weaviate/
Pinecone) belongs to `../vector-db/SKILL.md`. Use that if your store fuses natively; use the RRF
above when you fuse two independent indexes yourself.
## 3. Rerank funnel
Funnel parameters (Microsoft Cloud Blog 2025-02-04; StackAI, accessed 2026-06-02):
| Param | Value | Why |
|---|---|---|
| retrieve top_n | ~150 | wide enough that the right chunk is almost always present |
| rerank model | Cohere Rerank 3.5 (ctx 4096) or local cross-encoder | cross-encoder scores query+chunk jointly |
| keep top_k | ~20 | precise enough for the prompt without diluting it |
```python
import cohere
co = cohere.ClientV2()
def rerank(query, candidates, top_k=20):
res = co.rerank(model="rerank-v3.5", query=query,
documents=[c["text"] for c in candidates], top_n=top_k)
return [candidates[r.index] for r in res.results] # ids preserved by index
```
Local alternative: load a `bge-reranker`-class cross-encoder and score each `(query, chunk)`
pair; sort descending; slice top_k. Use it when data cannot leave the network.
## 4. The grounding system prompt (complete)
Three non-negotiable clauses: answer only from context, cite chunk ids, explicit refusal.
```text
You are a retrieval-grounded assistant. You answer ONLY using information inside the <context>
block below. Do not use prior knowledge or make assumptions.
Rules:
1. Every factual claim MUST cite the chunk id it came from, written inline as [chunk_id].
You may cite multiple ids for one claim.
2. If the <context> does not contain enough information to answer the question, reply EXACTLY:
"I don't have enough information in the provided sources to answer that."
(For Spanish corpora: "No tengo suficiente información en las fuentes para responder.")
3. Do not fabricate citations. If you cannot cite it, do not state it.
<context>
{{#each chunks}}[{{chunk_id}}] {{text}}
{{/each}}
</context>
Question: {{question}}
```
## 5. Minimal runnable skeleton
```python
# retrieve -> rerank -> ground -> answer. Provider-neutral glue; swap clients as needed.
def answer(query):
qv = embed(query) # same model as the corpus
dense = vector_store.search(qv, limit=150) # ../vector-db
keyword = bm25.search(query, limit=150)
fused = rrf(dense, keyword, top_n=150)
top = rerank(query, fused, top_k=20) # section 3
context = "\n".join(f"[{c['chunk_id']}] {c['text']}" for c in top)
resp = llm.messages.create(
model="claude-sonnet-4-5",
max_tokens=1024,
system=GROUNDING_PROMPT, # section 4, refusal clause included
messages=[{"role": "user",
"content": f"<context>\n{context}\n</context>\n\nQuestion: {query}"}],
)
return resp.content[0].text # contains [chunk_id] citations or the refusal line
```
Keep `chunk_id` + `source` on every object from chunking through this function so the cited ids
in the answer map back to real source pages.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack." | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |