Skip to content

mikeparcewski/wicked-garden / skills/agentic/context-engineering

wicked-garden-agentic-context-engineering

Verified

Context window management, token optimization, and memory patterns for efficient multi-agent systems. Use when: optimizing token usage in an agentic pipeline, designing memory scope for short / long-term / episodic state, or applying a context-loading strategy (anticipatory / JIT / hybrid).

refs/compression-techniques.md

View on GitHub
---
phase_relevance: ["design", "review"]
archetype_relevance: ["*"]
---
# Context Optimization Techniques

Advanced strategies for minimizing token usage while maximizing context quality.

## Compression Techniques

### 1. Conversation Summarization

Progressive summarization of conversation history.

```python
class ConversationCompressor:
    def __init__(self, llm, max_full_messages=10, summary_length=500):
        self.llm = llm
        self.max_full_messages = max_full_messages
        self.summary_length = summary_length

    async def compress(self, messages):
        if len(messages) <= self.max_full_messages:
            return messages

        # Split into old (to summarize) and recent (keep full)
        old_messages = messages[:-self.max_full_messages]
        recent_messages = messages[-self.max_full_messages:]

        # Create summary of old messages
        summary_prompt = f"""Summarize this conversation in {self.summary_length} words or less.
        Focus on:
        - Key decisions made
        - Important facts established
        - Current task state

        Conversation:
        {self.format_messages(old_messages)}
        """

        summary = await self.llm.generate(summary_prompt, max_tokens=self.summary_length * 2)

        # Return summary + recent messages
        return [
            {"role": "system", "content": f"Previous conversation summary:\n{summary}"}
        ] + recent_messages

    def format_messages(self, messages):
        return "\n".join(f"{m['role']}: {m['content']}" for m in messages)
```

### 2. Semantic Deduplication

Remove redundant information.

```python
from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity

class SemanticDeduplicator:
    def __init__(self, similarity_threshold=0.9):
        self.encoder = SentenceTransformer('all-MiniLM-L6-v2')
        self.threshold = similarity_threshold

    def deduplicate(self, texts):
        if not texts:
            return texts

        # Encode all texts
        embeddings = self.encoder.encode(texts)

        # Keep track of which to keep
        keep_indices = [0]  # Always keep first

        for i in range(1, len(texts)):
            # Compare to all kept texts
            similarities = cosine_similarity(
                [embeddings[i]],
                embeddings[keep_indices]
            )[0]

            # Only keep if not too similar to existing
            if max(similarities) < self.threshold:
                keep_indices.append(i)

        return [texts[i] for i in keep_indices]

# Usage
deduper = SemanticDeduplicator()
unique_contexts = deduper.deduplicate([
    "The user prefers Python",
    "User likes Python programming",  # Very similar - would be removed
    "The user works at Acme Corp"      # Different - would be kept
])
```

### 3. Hierarchical Summarization

Summarize at multiple levels of detail.

```python
class HierarchicalSummarizer:
    async def summarize(self, text, levels=3):
        summaries = {}

        # Level 0: Full text
        summaries['full'] = text

        # Level 1: Detailed summary (50% reduction)
        summaries['detailed'] = await self.llm.generate(
            f"Summarize in half the length:\n{text}"
        )

        # Level 2: Brief summary (80% reduction)
        summaries['brief'] = await self.llm.generate(
            f"Summarize in 1-2 paragraphs:\n{summaries['detailed']}"
        )

        # Level 3: Ultra-brief (95% reduction)
        summaries['headline'] = await self.llm.generate(
            f"Summarize in one sentence:\n{summaries['brief']}"
        )

        return summaries

# Usage: Choose appropriate level based on available tokens
summaries = await summarizer.summarize(long_document)
if tokens_available > 5000:
    context = summaries['detailed']
elif tokens_available > 1000:
    context = summaries['brief']
else:
    context = summaries['headline']
```

### 4. Entity-Based Compression

Extract and store only key entities and relationships.

```python
class EntityCompressor:
    async def compress(self, text):
        # Extract entities
        entities_prompt = f"""Extract key entities and relationships from this text.
        Format as JSON: {{"entities": [...], "relationships": [...]}}

        Text: {text}
        """

        entities = await self.llm.generate(entities_prompt)

        # Store compressed representation
        return {
            'entities': entities['entities'],
            'relationships': entities['relationships'],
            'original_length': len(text),
            'compressed_length': len(str(entities))
        }

    def reconstruct_context(self, compressed):
        # Build minimal context from entities
        context = "Key facts:\n"
        for entity in compressed['entities']:
            context += f"- {entity}\n"

        context += "\nRelationships:\n"
        for rel in compressed['relationships']:
            context += f"- {rel}\n"

        return context
```
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall