Skip to content

Unique-AG/ai / .claude/skills/model-deployment

model-deployment

Verified

Deploy a new LLM model via LiteLLM (GitOps) and/or Azure (Terraform + node backend), add it to the ai toolkit, create the required pull requests, and verify end-to-end. Use when rolling out a model to any environment, adding a model to the toolkit, troubleshooting "model not available" errors, or finding pricing/token limits/model ids for any provider.

references/LITELLM-CONFIG.md

View on GitHub
# Track A — LiteLLM config details

## Config entry examples

From our actual ArgoCD overlays:

```yaml
# Standard model
- model_name: anthropic-claude-opus-4-6
  litellm_params:
    model: anthropic/claude-opus-4-6
    api_key: os.environ/ANTHROPIC_API_KEY
    max_tokens: 20000

# Gemini model
- model_name: gemini-2-5-flash
  litellm_params:
    model: gemini/gemini-2.5-flash
    api_key: os.environ/GEMINI_API_KEY

# Model with thinking/reasoning enabled
- model_name: anthropic-claude-3-7-sonnet-thinking
  litellm_params:
    model: anthropic/claude-3-7-sonnet-20250219
    api_key: os.environ/ANTHROPIC_API_KEY
    temperature: 1.0
    max_tokens: 64000
    thinking:
      type: enabled
      budget_tokens: 20000
    additional_drop_params: ["top_p"]
```

### Notes

- `api_key` uses `os.environ/<KEY_NAME>` syntax — LiteLLM resolves this at runtime from pod environment.
- `max_tokens` is the **output** token limit for the proxy; set it conservatively to avoid exceeding context limits.
- For thinking/reasoning models, set `temperature: 1.0` (required by the provider) and configure the `thinking` block.

## Provider prefix conventions

- Gemini API key auth: `gemini/<model>` with `GEMINI_API_KEY`
- Anthropic: `anthropic/<model>` with `ANTHROPIC_API_KEY`
- OpenAI via LiteLLM: `openai/<model>` with `OPENAI_API_KEY`
- If the LiteLLM registry shows a different prefix for a new provider, confirm with the team first.

## model_name vs litellm_params.model

- `model_name` = user-facing key (no provider prefix): `gemini-2-5-flash`
- `litellm_params.model` = provider model string (with prefix): `gemini/gemini-2.5-flash`
- Common mistake: putting a provider string as `model_name`. If `model_name` contains `/`, it is almost certainly wrong.

## Preview naming

If the upstream model uses `-preview` in its name, keep `-preview` in the exposed `model_name` too.

## Troubleshooting

| Symptom | Likely cause |
|---------|-------------|
| Model not visible in LiteLLM UI | ArgoCD app not synced, Deployment not rolled, or YAML syntax error |
| "Model not available on cluster" | Config not applied, pods running old config, or model name mismatch |
| Wrong responses / unexpected model | Wrong provider prefix (e.g. missing `gemini/` prefix with `GEMINI_API_KEY`) |

## ArgoCD vs LiteLLM dashboard

- **ArgoCD** is the source of truth for config sync ("sync from git" happens here).
- **LiteLLM dashboard** is for inspecting proxy state (model list, logs) — you cannot deploy or sync from it.
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall