Skip to content

ericrisco/rsc-harness / skills/llm-pipeline

llm-pipeline

Unverified

Use when wiring several LLM calls into a production flow: typed contracts between steps, a router/gateway so 429s, timeouts and outages fail over instead of taking you down, and cost control via caching, model tiers and abort caps. NOT single-prompt wording (that is `prompt-engineering`), NOT a model-driven tool loop (that is `building-agents`).

FileKindSize
SKILL.mdskill md11.8 KB
evals/README.mdmarkdown881 B
evals/cases.yamlconfig3.4 KB
references/caching-layers.mdmarkdown3.0 KB
references/litellm-router.mdmarkdown3.9 KB
scripts/verify.shexecutablescript6.5 KB

references/litellm-router.md

View on GitHub
# LiteLLM Router / Proxy config (v1.83.x)

Fuller config than the SKILL body. Version-pinned to the **v1.83.3-stable** line (also recent: v1.81.9-stable, v1.81.13). LiteLLM exposes a unified OpenAI-format `completion()` across 100+ providers, with retry/fallback, cost tracking, and budget management, as an SDK or a Proxy Server.

## SDK Router with every fallback bucket

```python
from litellm import Router

router = Router(
    model_list=[
        {"model_name": "smart",
         "litellm_params": {"model": "anthropic/claude-sonnet-4-6",
                            "api_key": "os.environ/ANTHROPIC_API_KEY",
                            "timeout": 30, "order": 1}},
        {"model_name": "smart",          # same group, order 2 = first internal fallback
         "litellm_params": {"model": "openai/gpt-4o",
                            "api_key": "os.environ/OPENAI_API_KEY",
                            "timeout": 30, "order": 2}},
        {"model_name": "cheap",
         "litellm_params": {"model": "anthropic/claude-haiku-4-5",
                            "api_key": "os.environ/ANTHROPIC_API_KEY",
                            "timeout": 20}},
        {"model_name": "long-ctx",
         "litellm_params": {"model": "openai/gpt-4o",
                            "api_key": "os.environ/OPENAI_API_KEY", "timeout": 60}},
    ],
    # cross-group fallbacks (after order levels in a group are exhausted)
    fallbacks=[{"smart": ["cheap"]}],
    content_policy_fallbacks=[{"smart": ["smart"]}],   # ContentPolicyViolationError
    context_window_fallbacks=[{"smart": ["long-ctx"]}],# ContextWindowExceededError
    default_fallbacks=["cheap"],                         # catch-all last resort
    num_retries=2,            # bounded retries PER order level
    timeout=30,              # router-wide hard cap
    retry_after=1,
)
```

Order semantics: a request to an `order=1` deployment that fails (connection error, 404, 429, ...) auto-tries `order=2`, then `order=3`. Each order level runs its own `num_retries` before escalating. Once all orders in the group are exhausted, the `fallbacks`/specialized buckets fire. Retry backoff is exponential from `INITIAL_RETRY_DELAY` 0.2s up to `MAX_RETRY_DELAY` 10s, with jitter.

## Keys come from env, never inline

Use `os.environ/VAR` indirection (proxy YAML) or read `os.environ[...]` in SDK. A literal `sk-...` in a model_list is a leaked credential and a verify.sh FAIL.

## Proxy Server config (config.yaml)

```yaml
model_list:
  - model_name: smart
    litellm_params:
      model: anthropic/claude-sonnet-4-6
      api_key: os.environ/ANTHROPIC_API_KEY
      timeout: 30
  - model_name: smart-backup
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY
      timeout: 30

router_settings:
  num_retries: 2
  timeout: 30
  fallbacks: [{"smart": ["smart-backup"]}]
  context_window_fallbacks: [{"smart": ["smart-backup"]}]

litellm_settings:
  cache: true
  cache_params:
    type: redis
    host: os.environ/REDIS_HOST
    port: os.environ/REDIS_PORT
    ttl: 600
```

## Budget & rate limits

```yaml
litellm_settings:
  max_budget: 100         # $ hard cap across the proxy; aborts over-budget requests
  budget_duration: 30d
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
```

Per-key / per-tenant budgets are set on virtual keys (`/key/generate` with `max_budget` and `tpm_limit`/`rpm_limit`). Budgets that ABORT are the point — a logged-only budget does not stop a runaway loop.

## Cost callback

```python
import litellm

def track(kwargs, response, start, end):
    cost = kwargs.get("response_cost")
    # emit per-step: model, tokens, cost, latency, cache_hit, fallback used
    log.info("llm_call", model=kwargs["model"], cost=cost,
             latency_ms=(end - start).total_seconds() * 1000)

litellm.success_callback = [track]
```

The cost/latency numbers belong in your observability backbone; attribution dashboards belong to the cost-tracking skill, not here.
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall