agentsope/SkillAlchemy / skills/agentsop-multiscale-chunking
agentsop-multiscale-chunking
VerifiedDesigns multiscale chunking for RAG by embedding small units for retrieval precision and returning larger context for synthesis. Use when fixed-size chunks either lose surrounding context or dilute relevance in long documents, manuals, filings, or codebases. Covers sentence-window and parent-child or auto-merging strategies, base chunk sizing, and evaluation. Do not use when a single chunk scale already meets retrieval and generation needs.
| File | Kind | Size |
|---|---|---|
| README.md | markdown | 1.9 KB |
| SKILL.md | skill md | 19.5 KB |
| intermediate/operation_candidates.json | config | 6.2 KB |
| references/R1-source-evidence.md | markdown | 7.3 KB |
README.md
View on GitHub# multiscale-chunking (C5 enhancement overlay)
Enhancement overlay on top of [[llamaindex]] that resolves the **chunk
paradox** in RAG over long documents: small chunks win retrieval precision but
lose context, large chunks win context but dilute embedding relevance into
"topic averages". The base LlamaIndex skill lists `DecoupleChunkScope` as one
optimization knob among many; this overlay turns that single knob into a full
recipe.
**Core flip**: decouple the embed-unit from the return-unit — *embed small for
retrieval precision, return large for generation context*.
**What it adds**
- A three-gate SOP: prove a single chunk size is insufficient (sweep) → choose a
horizontal (sentence-window) vs vertical (auto-merging / parent-child)
expansion strategy → measure the lift or revert.
- 7 operation models (MSC-01…07) refining LlamaIndex OP-02 / OP-05 into
executable sub-steps.
- 2 dilemma cases: sentence-window vs auto-merging, and the chunk-size sweep
(Uber 10-K → 1024 prose optimum; when it does *not* converge, decouple).
- Anti-patterns (the `chunk_size=4096` reflex, sentence-window without
metadata replacement, indexing parent nodes) and boundaries (short corpora,
hard real-time).
- Cross-framework map: LlamaIndex `AutoMergingRetriever` / `HierarchicalNodeParser`
/ `SentenceWindowNodeParser` ≈ LangChain `ParentDocumentRetriever`.
**Activate when** a chunk-size sweep has stalled (no single size wins both
faithfulness and recall) over long prose / filings / manuals / code. **Do not**
activate for short static corpora or when the sweep already converged.
**Files**
- `SKILL.md` — the overlay (7 sections).
- `references/R1-source-evidence.md` — provenance for every claim.
- `intermediate/operation_candidates.json` — machine-readable MSC ops.
This overlay does not replace [[llamaindex]]; defer to the base skill for
baseline, eval loop, hybrid search, reranking, routing, and production hardening.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| brainstormingobra/superpowersYou MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation). | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| brand-guidelinesanthropics/skillsApplies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| frontend-designanthropics/skillsGuidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| mcp-builderanthropics/skillsGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK). | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |