Skip to content

Unique-AG/ai / .claude/skills/model-deployment

model-deployment

Verified

Deploy a new LLM model via LiteLLM (GitOps) and/or Azure (Terraform + node backend), add it to the ai toolkit, create the required pull requests, and verify end-to-end. Use when rolling out a model to any environment, adding a model to the toolkit, troubleshooting "model not available" errors, or finding pricing/token limits/model ids for any provider.

references/LESSONS-LEARNED.md

View on GitHub
# Lessons learned

These come from real deployment experience and real incidents — refer back to them when something looks off.

## `default_options` pitfall: `None` vs `"none"` (caused production incident)

When defining `default_options` for a reasoning model, **never** use Python `None` for `reasoning_effort`:

```python
# BAD — caused reasoning_effort=null sent to Azure, which rejects it
default_options={"reasoning_effort": None}

# GOOD — explicit string value, accepted by Azure
default_options={"reasoning_effort": "none"}

# ALSO GOOD — omit the key entirely if no default is needed
default_options={}
```

**What happened:** `AZURE_GPT_51_2025_1113` was defined with `"reasoning_effort": None` in `default_options`. The toolkit's `_prepare_responses_params_util` checked `"reasoning_effort" in model_info.default_options` (key existence), found the key, and constructed `Reasoning(effort=None)`. This serialized to `reasoning_effort: null` in the Azure API request. Azure rejected it: *"reasoning_effort does not support null. Supported values are: none, low, medium, high."*

The guard only checked if the key was present, not whether the value was non-null. The fix (v1.57.0) added `resolve_temp_and_reasoning()` which explicitly substitutes `None` with the model's declared default.

## Azure `sku_capacity` is a live quota, not a platform-wide constant

A requested quota (Global Standard 15000, Data Zone Standard 5000) is the ceiling. Each tenant's `sku_capacity` must fit the live Azure limit for that **subscription, region, and SKU**. Deployments that share a subscription and region share one bucket: unique dev and unique qa/lz both use Sweden Central, so two rows of 15000 ask for 30000 against a 15000 limit and Atlantis apply fails.

Read the limit with `az cognitiveservices usage list` before writing the number. Clamp to the limit, and split a shared bucket the way the sibling model already does. The procedure is in [AZURE-NODE-CHAT.md](AZURE-NODE-CHAT.md#azure-quota-sku_capacity).

## Dual code paths: node-chat vs assistants-core

Azure models are called through **two independent paths**:

1. **node-chat (TypeScript)** — Chat Completions and Responses API via the Azure SDK. Used by plain chat spaces.
2. **assistants-core (Python)** — Responses API via `unique_toolkit`. Used by agentic spaces with Python modules.

These paths have different handling for parameters like `reasoning_effort`. A bug in the toolkit Python path may not reproduce in node-chat, and vice versa. **Always test both paths** when deploying or updating a model: send a message in a plain chat space (node-chat) AND in an agentic assistant space (assistants-core).

## Cherry-pick completeness across repos

When cherry-picking model support into a release branch, **all cross-repo dependencies must be updated together**. In a past incident, GPT-5.4 was cherry-picked into the release branch (node-chat TypeScript changes) but the **monorepo** pin to the new published `unique_toolkit` wheel in `assistants-core` was forgotten. This left the buggy toolkit version in place for gpt-5.1, causing the `reasoning_effort: null` production error.

This is **not** a manual ai-repo `CHANGELOG.md` / `version` edit — after the toolkit ships from the ai repo, bump the **dependency constraint** in the monorepo (e.g. `assistants-core` `pyproject.toml` or the monorepo's AI version sync workflow).

Checklist for cherry-picks:
- [ ] AI repo: cherry-pick from `main` onto `release/YYYY.WW` and **rebase-merge** the PR (squash breaks release-please changelogs; see `hotfix-backport` skill)
- [ ] Node-chat TypeScript changes (language-model enum, factory, service)
- [ ] Monorepo: `assistants-core` (or equivalent) pins the new published `unique_toolkit` version
- [ ] LiteLLM overlay changes (if applicable)
- [ ] Terraform changes (if applicable)

## Responses API is stricter than Chat Completions API

- The Responses API (used by the Python toolkit in `assistants-core`) sends parameters directly to Azure and rejects invalid values like `reasoning_effort: null`.
- The Chat Completions API (used by node-chat via the Azure SDK) may silently drop or ignore malformed optional parameters in some cases.
- This means a model definition bug can cause errors in agentic Python spaces but not in plain chat spaces. When investigating "model X works in chat but fails in assistants," look at the toolkit's parameter construction.

## Restart the right thing

- To reload config/secrets: roll the **Kubernetes Deployment**.
- Restarting an **HPA** does not reload running pods (it only adjusts autoscaling).

## Never guess model facts

- Token limits, capabilities, model identifiers: use authoritative sources only.
- If no source is available, ask the user to provide a link or confirm explicitly.
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall