dominodatalab/domino-ai-plugin / skills/domino-llm-gateway
domino-llm-gateway
VerifiedCall Large Language Models through Domino LLM Gateway 2.0, the centrally governed gateway extension an administrator installs in front of external providers and Domino-hosted models. Covers finding and detecting the gateway, authentication with the in-run access token (fresh per call) or a Personal Access Token outside Domino, model aliases and model discovery, the OpenAI-compatible and Anthropic-compatible paths from the OpenAI and Anthropic SDKs, Pydantic AI, LangChain and plain HTTP, streaming, attribution headers, refused calls (login redirect, grants, budgets, guardrails), pointing a coding assistant such as Claude Code or Codex at the gateway, and what the administrator configures. Use when a user mentions LLM Gateway, Gateway 2.0, a gateway alias or URL, asks how to call a model through the Domino gateway, how to run a coding assistant through it, why a gateway call was refused, or what the gateway administrator must set up.
Domino LLM Gateway 2.0
Applies to Domino 6.3 and Domino Cloud where an administrator has installed LLM Gateway 2.0. It is an extension: installed separately, updated out of band, so whether it exists depends on the installation, not on the Domino version. It runs as a Domino App. If nobody has installed it, say so and use the domino-llm-access skill (a Domino-hosted endpoint or a provider SDK); do not try to install it.
What the gateway is
One base URL in front of many providers (external LLM vendors and Domino-hosted model endpoints). Provider keys stay in the gateway; callers authenticate with their Domino identity. Every request is tracked (user, Project, model, tokens, latency, estimated cost); administrators grant access per model alias, set budgets, and screen prompts and responses with guardrails.
Find and detect the gateway
The extension injects nothing into a Workspace: no URL, no token, no ANTHROPIC_* or OPENAI_* variable. Use LLM_GATEWAY_URL if the user or an administrator created it as a Project or user variable (not a DOMINO_-prefixed name; Domino reserves that prefix), else ask for the gateway App's URL (https://<domino or apps host>/apps/<app id or vanity URL>). Confirm it with the bearer: GET <base>/v1/models returns JSON and GET <base>/health returns {"status": "ok"}. HTML proves nothing: with a bearer <base>/ is the gateway's UI (200); without one every route is a 302 to the Domino login page.
| Family | Path | Use with |
|---|---|---|
| OpenAI-compatible | <base>/v1 (/models, /chat/completions, /responses; /embeddings where an embedding alias exists) | OpenAI SDK, Pydantic AI, LangChain, any OpenAI-style client |
| Anthropic-compatible | <base>/anthropic (/v1/messages) | Anthropic SDK, Claude Code |
Authentication
Send Authorization: Bearer <token> on every request.
- Inside a Domino Workspace, Job or App: read the access token from
$DOMINO_API_PROXY/access-token(by defaulthttp://localhost:8899/access-token). It is short-lived, so re-read it before every request instead of caching it in a long-running process. In an App it is the publisher's token: every viewer's calls use the publisher's grants and budget, so tag them withX-LLM-Tag-<key>to attribute usage. - Outside Domino (laptop, CI): a Domino bearer such as a Personal Access Token in a variable the user creates, for example
LLM_GATEWAY_TOKEN. A Domino access token works from a laptop; a PAT or one of the gateway's own tokens (prefixdgw_, issued by an administrator) may be answered with the App login redirect or 401, in which case use a Domino access token. - Legacy user API keys are deprecated and not used. The Anthropic path needs the bearer in
Authorization; a credential sent only asx-api-keyis treated as no credential.
import os
import requests
from openai import OpenAI # not in the Domino Standard Environment: pip install openai
GATEWAY = os.environ["LLM_GATEWAY_URL"].rstrip("/")
def token() -> str:
proxy = os.environ.get("DOMINO_API_PROXY")
if proxy: # inside Domino: fresh per call
return requests.get(f"{proxy}/access-token", timeout=5).text.strip()
return os.environ["LLM_GATEWAY_TOKEN"] # outside: a Domino bearer the user stored
def chat(messages, model="domino/example-alias", **kwargs) -> str: # an alias from /v1/models
client = OpenAI(base_url=f"{GATEWAY}/v1", api_key=token())
return client.chat.completions.create(model=model, messages=messages, **kwargs).choices[0].message.content
Models
model is an alias from GET <base>/v1/models (OpenAI list shape, data[].id). Aliases are administrator-defined and may carry a prefix such as domino/; an alias's vendor name may also be accepted, but only listed aliases are guaranteed. GET <base>/v1/whoami shows how the gateway resolved your identity.
The openai, anthropic, pydantic-ai and langchain-openai packages are not in the Domino Standard Environment: pip install them in a Workspace to try; add them to the Compute Environment's Dockerfile for repeat use.
Call it
- OpenAI SDK, Responses API, embeddings, Pydantic AI, LangChain, plain HTTP and streaming: CALLING-MODELS.md.
- Anthropic SDK:
Anthropic(base_url=f"{GATEWAY}/anthropic", auth_token=token()), thenclient.messages.create(model="<alias>", ...);api_key=silently breaks (see errors). - Attribution:
X-LLM-Tag-<key>: <value>headers (for exampleX-LLM-Tag-team: research) slice usage; they are labels, not access control. - Tracing: OpenAI and Anthropic SDK calls are autologged by
mlflow.openai.autolog()/mlflow.anthropic.autolog(), so thedomino-genai-tracingskill works unchanged.
When a call is refused
A 302 to the Domino login page (HTML) is a missing, malformed or expired bearer: re-read the access token per call. A 302 to /app-consent?appId=... means the user has not consented to the gateway App acting for them: open the gateway URL in a browser once and accept. The Anthropic SDK with api_key= gets that HTML as a string and fails with AttributeError: 'str' object has no attribute 'content'; use auth_token=. 404 "Model '...' not found" is an unknown alias: list /v1/models. A 4xx naming a grant, budget or guardrail rule means no access, a spent budget, or a denied prompt or response: ask the administrator, or change the prompt. A 403 such as Only a gateway admin can manage gateway admins is an administrator-only route. 413 from the App proxy means the body is too large. Details: CALLING-MODELS.md.
Coding assistants through the gateway
Offer the user the choice; never route by default. Claude Code and Codex through the gateway, and the rule: CODING-ASSISTANTS.md.
What the administrator configures
Providers, model aliases, access grants and budgets, guardrails, gateway tokens, and the usage and audit views, all in the gateway UI: ADMIN-OVERVIEW.md. Installing and updating the extension is the administrator's domain.
Documentation
- LLM Gateway 2.0: https://docs.domino.ai/cloud/platform-capabilities/features/llms/llm-gateway
- Set up LLM access: https://docs.domino.ai/cloud/platform-capabilities/features/llms (6.3: replace
cloudwith6.3) - Authentication: https://docs.domino.ai/cloud/reference/api/domino-api-authentication
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |