Skip to content

YemenJPT/YemenJPT-AI-Skills / skills/core/azure-ai-voicelive-py

azure-ai-voicelive-py

Verified

Build real-time voice AI applications using Azure AI Voice Live SDK (azure-ai-voicelive). Use this skill when creating Python applications that need real-time bidirectional audio communication with Azure AI, including voice assistants, voice-enabled chatbots, real-time speech-to-speech translation, voice-driven avatars, or any WebSocket-based audio streaming with AI models. Supports Server VAD (Voice Activity Detection), turn-based conversation, function calling, MCP tools, avatar integration, and transcription.

FileKindSize
README.ar.mdmarkdown4.5 KB
README.en.mdmarkdown3.2 KB
SKILL.mdskill md13.7 KB
references/api-reference.mdmarkdown8.7 KB
references/examples.mdmarkdown16.9 KB
references/models.mdmarkdown18.0 KB

references/api-reference.md

View on GitHub
# Azure AI Voice Live SDK - API Reference

## Table of Contents
- [connect() Function](#connect-function)
- [VoiceLiveConnection](#voiceliveconnection)
- [SessionResource](#sessionresource)
- [ResponseResource](#responseresource)
- [InputAudioBufferResource](#inputaudiobufferresource)
- [OutputAudioBufferResource](#outputaudiobufferresource)
- [ConversationResource](#conversationresource)
- [TranscriptionSessionResource](#transcriptionsessionresource)
- [WebsocketConnectionOptions](#websocketconnectionoptions)
- [Exceptions](#exceptions)

---

## connect() Function

Creates an async context manager for WebSocket connections.

```python
from azure.ai.voicelive.aio import connect

async with connect(
    credential: Union[AzureKeyCredential, AsyncTokenCredential],
    endpoint: str,
    api_version: str = "2025-10-01",
    model: Optional[str] = None,
    query: Optional[Mapping[str, Any]] = None,
    headers: Optional[Mapping[str, Any]] = None,
    connection_options: Optional[WebsocketConnectionOptions] = None,
    credential_scopes: Optional[List[str]] = None,
    **kwargs
) -> VoiceLiveConnection:
    ...
```

### Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `credential` | `AzureKeyCredential` or `AsyncTokenCredential` | Yes | Authentication credential |
| `endpoint` | `str` | Yes | Service endpoint URL |
| `api_version` | `str` | No | API version (default: "2025-10-01") |
| `model` | `str` | Sometimes | Model identifier (required unless using Agent scenario) |
| `query` | `Mapping[str, Any]` | No | Additional URL query parameters |
| `headers` | `Mapping[str, Any]` | No | Additional HTTP headers |
| `connection_options` | `WebsocketConnectionOptions` | No | WebSocket transport options |
| `credential_scopes` | `List[str]` | No | OAuth scopes (default: `["https://ai.azure.com/.default"]`) |

---

## VoiceLiveConnection

Main connection class with resource accessors.

### Properties

| Property | Type | Description |
|----------|------|-------------|
| `session` | `SessionResource` | Session configuration |
| `response` | `ResponseResource` | Response management |
| `input_audio_buffer` | `InputAudioBufferResource` | Audio input buffer |
| `output_audio_buffer` | `OutputAudioBufferResource` | Audio output buffer |
| `conversation` | `ConversationResource` | Conversation items |
| `transcription_session` | `TranscriptionSessionResource` | Transcription config |

### Methods

```python
async def recv() -> ServerEvent:
    """Receive and parse the next typed server event."""

async def recv_bytes() -> bytes:
    """Receive raw bytes from the connection."""

async def send(event: Union[Mapping[str, Any], ClientEvent]) -> None:
    """Send an event to the server."""

async def close(*, code: int = 1000, reason: str = "") -> None:
    """Close the WebSocket connection."""

async def __aiter__() -> AsyncIterator[ServerEvent]:
    """Iterate over server events until connection closes."""
```

---

## SessionResource

Manage session configuration.

### Methods

```python
async def update(
    *,
    session: Union[Mapping[str, Any], RequestSession],
    event_id: Optional[str] = None
) -> None:
    """Update session configuration."""
```

### RequestSession Fields

| Field | Type | Description |
|-------|------|-------------|
| `instructions` | `str` | System prompt for the model |
| `modalities` | `List[Modality]` | `["text"]`, `["audio"]`, or `["text", "audio"]` |
| `voice` | `Voice` | Voice for audio output |
| `input_audio_format` | `InputAudioFormat` | Audio input format |
| `output_audio_format` | `OutputAudioFormat` | Audio output format |
| `turn_detection` | `TurnDetection` | VAD configuration or `None` for manual |
| `tools` | `List[Tool]` | Function tools |
| `tool_choice` | `ToolChoice` | `"auto"`, `"none"`, `"required"`, or specific function |
| `temperature` | `float` | Model temperature (0.6-1.2) |
| `max_response_output_tokens` | `int` or `"inf"` | Max tokens per response |
| `input_audio_transcription` | `AudioInputTranscriptionOptions` | Transcription settings |

---

## ResponseResource

Manage model responses.

### Methods

```python
async def create(
    *,
    response: Optional[Union[ResponseCreateParams, Mapping[str, Any]]] = None,
    event_id: Optional[str] = None,
    additional_instructions: Optional[str] = None
) -> None:
    """Create a response (trigger inference)."""

async def cancel(
    *,
    response_id: Optional[str] = None,
    event_id: Optional[str] = None
) -> None:
    """Cancel an in-progress response."""
```

### ResponseCreateParams Fields

| Field | Type | Description |
|-------|------|-------------|
| `modalities` | `List[Modality]` | Override session modalities |
| `instructions` | `str` | Override session instructions |
| `voice` | `Voice` | Override session voice |
| `temperature` | `float` | Override temperature |
| `max_response_output_tokens` | `int` | Override max tokens |
| `conversation` | `str` | `"auto"` or `"none"` |

---

## InputAudioBufferResource

Manage audio input buffer.

### Methods

```python
async def append(
    *,
    audio: str,  # Base64-encoded audio
    event_id: Optional[str] = None
) -> None:
    """Append audio data to the input buffer."""

async def commit(
    *,
    event_id: Optional[str] = None
) -> None:
    """Commit the buffer as a user message."""

async def clear(
    *,
    event_id: Optional[str] = None
) -> None:
    """Clear the input buffer without committing."""
```

---

## OutputAudioBufferResource

Manage audio output buffer.

### Methods

```python
async def clear(
    *,
    event_id: Optional[str] = None
) -> None:
    """Clear pending audio output (for interrupts)."""
```

---

## ConversationResource

Manage conversation state.

### Properties

| Property | Type | Description |
|----------|------|-------------|
| `item` | `ConversationItemResource` | Item operations |

---

## ConversationItemResource

CRUD operations on conversation items.

### Methods

```python
async def create(
    *,
    item: Union[ConversationRequestItem, Mapping[str, Any]],
    previous_item_id: Optional[str] = None,
    event_id: Optional[str] = None
) -> None:
    """Create a new conversation item."""

async def delete(
    *,
    item_id: str,
    event_id: Optional[str] = None
) -> None:
    """Delete a conversation item."""

async def retrieve(
    *,
    item_id: str,
    event_id: Optional[str] = None
) -> None:
    """Retrieve item details (server responds with event)."""

async def truncate(
    *,
    item_id: str,
    audio_end_ms: int,
    content_index: int,
    event_id: Optional[str] = None
) -> None:
    """Truncate audio at specified time."""
```

### ConversationRequestItem Types

```python
# User/Assistant/System message
{
    "type": "message",
    "role": "user" | "assistant" | "system",
    "content": [
        {"type": "input_text", "text": "..."},
        {"type": "input_audio", "audio": "base64..."},
        {"type": "text", "text": "..."},
        {"type": "audio", "audio": "base64...", "transcript": "..."}
    ]
}

# Function call (from model)
{
    "type": "function_call",
    "call_id": "...",
    "name": "function_name",
    "arguments": "{...}"
}

# Function output (from client)
{
    "type": "function_call_output",
    "call_id": "...",
    "output": "{...}"
}
```

---

## TranscriptionSessionResource

Configure input transcription.

### Methods

```python
async def update(
    *,
    session: Mapping[str, Any],
    event_id: Optional[str] = None
) -> None:
    """Update transcription session configuration."""
```

---

## WebsocketConnectionOptions

Transport configuration for the WebSocket connection.

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `compression` | `bool` or `int` | None | Enable per-message compression |
| `max_msg_size` | `int` | 4MB | Maximum message size |
| `heartbeat` | `float` | 30 | Keep-alive ping interval (seconds) |
| `autoclose` | `bool` | True | Auto-close on close frame |
| `autoping` | `bool` | True | Auto-respond to pings |
| `receive_timeout` | `float` | None | Message receive timeout |
| `close_timeout` | `float` | None | Close handshake timeout |
| `handshake_timeout` | `float` | None | Connection establishment timeout |
| `vendor_options` | `Mapping` | None | Implementation-specific options |

---

## Exceptions

```python
from azure.ai.voicelive.aio import ConnectionError, ConnectionClosed

class ConnectionError(AzureError):
    """Base WebSocket connection error."""

class ConnectionClosed(ConnectionError):
    """WebSocket connection was closed."""
    code: int      # Close code
    reason: str    # Close reason
```

### Common Close Codes

| Code | Meaning |
|------|---------|
| 1000 | Normal closure |
| 1001 | Going away |
| 1006 | Abnormal closure |
| 1008 | Policy violation |
| 1011 | Server error |
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 323kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall