Skip to content

alivirgo/Major-AI-Skills / plugins/mas-efficiency-pack/skills/cached-system-instructions

cached-system-instructions

Verified

Structure stable system instructions and variable input separately when configuring supported prompt-cache workflows.

Cached System Instructions (KV-Cache Prefix Architecture)

Overview

In Large Language Models, processing input tokens requires running dense matrix multiplications across the entire input sequence on every turn. In a 50-turn agent session with a 4,000-token system prompt, the provider re-processes the exact same 4,000 tokens 50 separate times ($200,000\text{ redundant token computations}$).

Prompt Caching (Anthropic Prompt Caching, OpenAI Automatic Prefix Caching, DeepSeek Context Caching) saves the transformer's Key-Value (KV) Activation States in GPU memory. Re-sending an identical prefix reuses the precomputed KV cache, cutting Time-to-First-Token (TTFT) latency by 80% and applying an automatic 90% cost discount.

The Hierarchical System Caching Protocol organizes instructions to guarantee uninterrupted prefix matching.


Uncached Re-computation vs. KV-Cache Hit

┌─────────────────────────────────────────────────────────────┐
│                 KV-Cache Mechanics Comparison               │
│                                                             │
│  Uncached Execution (Every Turn):                           │
│  • Re-computes Attention Matrices for 4,000 System Tokens   │
│  • TTFT Latency: 2.8 seconds                                │
│  • Full Input Rate: $3.00 / M tokens                        │
│                                                             │
│  KV-Cache Hit (Unbroken Static Prefix):                     │
│  • Directly loads KV-tensors from GPU VRAM                  │
│  • TTFT Latency: 0.35 seconds (8x Faster!)                  │
│  • Cached Input Rate: $0.30 / M tokens (90% Cost Discount)  │
└─────────────────────────────────────────────────────────────┘

The 4-Layer Caching Hierarchy

To prevent accidental cache invalidation, place instructions in order of decreasing stability:

┌───────────────────────────────────────────────────────────────────────────┐
│ LAYER 1: Base Agent Identity & Tool Schemas (100% Immutable) ──► CACHED   │
│ LAYER 2: Repository Architecture Blueprint (Static for Session)──► CACHED │
│ LAYER 3: MCP Tool Definitions & Skill Instructions         ──► CACHED     │
│ ───────────────────────────────────────────────────────────────────────── │
│ LAYER 4: Dynamic User Requests & Shell Outputs (Dynamic Tail)──► UNCACHED │
└───────────────────────────────────────────────────────────────────────────┘

Production Anthropic SDK Caching Implementation

To activate prompt caching in Anthropic Claude 3.5 Sonnet / Haiku, place cache_control breakpoints at the end of large static blocks (minimum threshold: 1,024 tokens on Sonnet, 2,048 tokens on Haiku):

import anthropic

client = anthropic.Anthropic()

SYSTEM_PROMPT = """
You are Antigravity, an elite autonomous software engineering agent.
... [3,500 tokens of static rules, AST guidelines, and tool schemas] ...
"""

def query_agent_with_caching(user_query: str, history: list) -> str:
    response = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=2048,
        system=[
            {
                "type": "text",
                "text": SYSTEM_PROMPT,
                # Set cache breakpoint on the static system prompt
                "cache_control": {"type": "ephemeral"}
            }
        ],
        messages=[
            *history,
            {"role": "user", "content": user_query}
        ]
    )
    
    # Inspect cache performance
    usage = response.usage
    print(f"Cache Created: {getattr(usage, 'cache_creation_input_tokens', 0)}")
    print(f"Cache Read (90% discount): {getattr(usage, 'cache_read_input_tokens', 0)}")
    return response.content[0].text

The 3 Golden Rules to Prevent Cache Busting

  1. Never Inject Timestamps into the System Header: Dynamic strings like "Current Time: 2026-08-22T17:52:00" change every second and bust the entire prefix cache. Pass current time in the User turn.
  2. Deterministic Tool Serialization: Ensure tool schemas are serialized with sorted JSON keys (json.dumps(obj, sort_keys=True)) to maintain bit-for-bit string equality.
  3. Keep the Head Static: Never insert dynamic project names at line 1. Put project names at the bottom of the system block or in the first user message.

Benchmark Metrics

Evaluation across a 30-turn autonomous feature build (4,000-token system prompt):

MetricUncached BaselineCached Prefix ArchitectureImprovement
Time-to-First-Token (TTFT)3.10 seconds0.38 seconds8.1x Faster Velocity
System Prompt Ingestion Cost$0.36$0.03690% Cost Reduction
Total Session Turnaround185 seconds48 seconds3.8x Total Speedup

Agent Operational Directive

MANDATORY: All system prompts exceeding 1,000 tokens must be structured as immutable, deterministic headers with prompt-cache control breakpoints enabled. Never inject dynamic runtime variables into the top-level system prefix.

reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness.Stars: 322kUpdated Claude CodeCodexVerifiedInstall
dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependenciesStars: 322kUpdated Claude CodeCodexVerifiedInstall
executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is availableStars: 322kUpdated Claude CodeCodexVerifiedInstall