Skip to content

cianfhoghlaim/cianchosaint / .agents/skills/baml

references/clients-and-retries.md

View on GitHub
# Named Clients + Retry Policies

BAML supports **named clients** with explicit `retry_policy` blocks,
`fallback` chains, and `round-robin` strategies. This is the
canonical way to manage LLM clients in BAML.

## The 3 Cianfhoghlaim production clients

The `baml/clients.baml` defines:

```baml
client<llm> ExtractEn {
  provider "openai"
  options {
    model "gpt-4o-mini"
    temperature 0.0
  }
  retry_policy Constant {
    max_retries 3
    delay_ms 1000
  }
}

client<llm> ExtractEnStrong {
  provider "openai"
  options {
    model "gpt-4o"
    temperature 0.0
  }
  retry_policy Exponential {
    max_retries 5
    strategy { type exponential_backoff, multiplier 2.0 }
  }
}

client<llm> LocalVision {
  provider "openai-generic"
  options {
    base_url "http://localhost:8000/v1"
    model "dots-ocr"
  }
  retry_policy Constant {
    max_retries 2
    delay_ms 500
  }
}
```

## Provider list

| Provider | `provider` value | Notes |
|:--|:--|:--|
| OpenAI (chat completions) | `"openai"` | Standard chat completions API |
| OpenAI (responses API) | `"openai-responses"` | Newer, supports built-in tools |
| Anthropic | `"anthropic"` | Claude 3.5 Sonnet, Claude 3 Opus |
| Google AI | `"google-ai"` | Gemini 2.5 Flash, Gemini 2.5 Pro |
| Vertex AI | `"vertex-ai"` | GCP-hosted Gemini / Claude |
| AWS Bedrock | `"aws-bedrock"` | Claude / Titan / Llama 2 |
| Azure OpenAI | `"azure-openai"` | Azure-hosted OpenAI |
| OpenAI generic | `"openai-generic"` | Any OpenAI-compatible API (vLLM, llama.cpp, etc.) |
| Ollama | `"ollama"` | Local model server |
| Anthropic Vertex | `"anthropic-vertex"` | Claude on Vertex AI |

## Retry policies

```baml
// Constant: retry every N ms, up to M times
retry_policy Constant {
  max_retries 3
  delay_ms 1000
}

// Exponential: backoff doubles each retry
retry_policy Exponential {
  max_retries 5
  strategy { type exponential_backoff, multiplier 2.0 }
  max_delay_ms 30000  // cap the delay at 30s
}

// Exponential with jitter (recommended for high-concurrency)
retry_policy ExponentialWithJitter {
  max_retries 5
  strategy { type exponential_backoff, multiplier 2.0, jitter true }
}
```

## Fallback chains

```baml
client<llm> PrimaryWithFallback {
  provider "openai"
  options { model "gpt-4o" temperature 0.0 }
  fallback [
    (provider "anthropic", options { model "claude-3-5-sonnet-20241022" temperature 0.0 }),
    (provider "google-ai", options { model "gemini-2.5-pro" temperature 0.0 }),
  ]
  retry_policy Constant { max_retries 2 delay_ms 500 }
}
```

When the primary fails, BAML falls through to the fallback clients
in order. Each fallback can have its own `retry_policy`.

## Round-robin (load balancing)

```baml
client<llm> RoundRobin {
  provider "openai"
  options { model "gpt-4o-mini" }
  round_robin [
    (provider "openai", options { model "gpt-4o-mini" api_key env.OPENAI_API_KEY_1 }),
    (provider "openai", options { model "gpt-4o-mini" api_key env.OPENAI_API_KEY_2 }),
    (provider "openai", options { model "gpt-4o-mini" api_key env.OPENAI_API_KEY_3 }),
  ]
}
```

Useful for distributing load across multiple API keys (rate-limit
avoidance).

## Multimodal client config (vision-specific)

```baml
client<llm> Gemini25Flash {
  provider "google-ai"
  options {
    model "gemini-2.5-flash"
    generationConfig {
      temperature 0.0
      maxOutputTokens 8192
    }
    safetySettings [
      { category "HARM_CATEGORY_HARASSMENT" threshold "BLOCK_NONE" }
      { category "HARM_CATEGORY_HATE_SPEECH" threshold "BLOCK_NONE" }
      { category "HARM_CATEGORY_SEXUALLY_EXPLICIT" threshold "BLOCK_NONE" }
      { category "HARM_CATEGORY_DANGEROUS_CONTENT" threshold "BLOCK_NONE" }
    ]
  }
}
```

## Using a named client in a function

```baml
function ExtractPrimaryLearningOutcomes(text: string) -> PrimaryLearningOutcome[] {
  client ExtractEn  // ← use the named client
  prompt #"..."#
}

function ExtractPrimaryLearningOutcomesStrong(text: string) -> PrimaryLearningOutcome[] {
  client ExtractEnStrong  // ← use the stronger client
  prompt #"..."#
}
```

## Async client mode

```baml
client<llm> ExtractEn {
  provider "openai"
  options { model "gpt-4o-mini" }
  default_client_mode async  // or "sync"
}
```

For high-concurrency Cianfhoghlaim pipelines (the dlt → BAML → Dagster
pattern), `default_client_mode async` is required.

## Cianfhoghlaim conventions

- `ExtractEn` (GPT-4o-mini) for production extraction
- `ExtractEnStrong` (GPT-4o) for higher-accuracy or fallback
- `LocalVision` (dots-ocr) for offline OCR
- `Gemini25Flash` (vision) for multimodal extraction
- Always `temperature 0.0` for deterministic extraction
- Always wrap the LLM in a `retry_policy` (default `Constant` with
  3 retries, 1s delay)
- For multimodal: `safetySettings` with `BLOCK_NONE` (Cianfhoghlaim content
  includes historical political / cultural material)

## Reference

- The full `baml/clients.baml` (and `clients_0.baml`
  in the repo) define the Cianfhoghlaim production clients
- BAML clients docs: <https://docs.boundaryml.com/docs/snippets/clients>
- BAML retry docs: <https://docs.boundaryml.com/docs/configuration/retries>
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall