Robertjam954/multimodal_rag_application / .claude/skills/azure-diagnostics
azure-diagnostics
VerifiedDebug Azure production issues on Azure using AppLens, Azure Monitor, resource health, and safe triage. WHEN: debug production issues, troubleshoot app service, app service high CPU, app service deployment failure, troubleshoot container apps, troubleshoot functions, troubleshoot AKS, kubectl cannot connect, kube-system/CoreDNS failures, pod pending, crashloop, node not ready, upgrade failures, analyze logs, KQL, insights, image pull failures, cold start issues, health probe failures, resource health, root cause of errors, troubleshoot event hubs, troubleshoot service bus, messaging SDK error, AMQP connection failure, message lock lost, service bus dead letter.
troubleshooting/messaging/azure-eventhubs-py.md
View on GitHub# Azure Event Hubs SDK — Python
Package: `azure-eventhub` | [README](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/eventhub/azure-eventhub) | [Full Troubleshooting Guide](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/eventhub/azure-eventhub/TROUBLESHOOTING.md)
## Common Errors
| Exception | Cause | Fix |
|-----------|-------|-----|
| `EventHubError` | Base exception wrapping AMQP errors | Check `message`, `error`, `details` fields |
| `ConnectionLostError` | Idle connection disconnected | Auto-recovers on next operation; no action needed |
| `AuthenticationError` | Bad credentials or expired SAS | Regenerate key, check RBAC roles, verify connection string |
| `OperationTimeoutError` | Network or throttling | Check firewall, try WebSockets (port 443), increase timeout |
## Retry Configuration
> **Auth:** `DefaultAzureCredential` is for local development. See [auth-best-practices.md](auth-best-practices.md) for production patterns.
```python
from azure.eventhub import EventHubProducerClient
from azure.identity import DefaultAzureCredential
client = EventHubProducerClient(
fully_qualified_namespace="<your-namespace>.servicebus.windows.net",
eventhub_name="<your-eventhub>",
credential=DefaultAzureCredential(),
retry_total=3,
retry_backoff_factor=0.8,
retry_backoff_max=120,
retry_mode='exponential'
)
```
## Consumer Client Retry Configuration
> **Auth:** `DefaultAzureCredential` is for local development. See [auth-best-practices.md](auth-best-practices.md) for production patterns.
Under heavy load, tune the retry policy on `EventHubConsumerClient` to reduce timeouts:
| Parameter | Default | Description |
|-----------|---------|-------------|
| `retry_total` | 3 | Max retry attempts per operation |
| `retry_backoff_factor` | 0.8 | Backoff multiplier between retries (seconds) |
| `retry_backoff_max` | 120 | Max backoff interval (seconds) |
| `retry_mode` | `exponential` | `fixed` or `exponential` |
```python
from azure.eventhub import EventHubConsumerClient
from azure.eventhub.extensions.checkpointstoreblob import BlobCheckpointStore
from azure.identity import DefaultAzureCredential
credential = DefaultAzureCredential()
checkpoint_store = BlobCheckpointStore(
blob_account_url="https://<storage-account>.blob.core.windows.net",
container_name="<checkpoint-container>",
credential=credential
)
client = EventHubConsumerClient(
fully_qualified_namespace="<your-namespace>.servicebus.windows.net",
eventhub_name="<your-eventhub>",
consumer_group="$Default",
credential=credential,
checkpoint_store=checkpoint_store,
retry_total=5,
retry_backoff_factor=1.0,
retry_backoff_max=120,
retry_mode='exponential'
)
```
## Enable Logging
```python
import logging, sys
handler = logging.StreamHandler(stream=sys.stdout)
handler.setFormatter(logging.Formatter("%(asctime)s | %(threadName)s | %(levelname)s | %(name)s | %(message)s"))
logger = logging.getLogger('azure.eventhub')
logger.setLevel(logging.DEBUG)
logger.addHandler(handler)
# Enable AMQP frame tracing
client = EventHubProducerClient(..., logging_enable=True)
```
## Key Issues
- **Buffered producer not sending**: Ensure enough `ThreadPoolExecutor` workers (one per partition). Use `buffer_concurrency` kwarg.
- **Blocking calls in async**: Run CPU-bound code in an executor; blocking the event loop impacts load balancing and checkpointing.
- **Consumer disconnected**: Expected during load balancing. If persistent with no scaling, file an issue.
- **Soft delete on checkpoint store**: Disable "soft delete" and "blob versioning" on the storage account used for checkpointing.
- **Always close clients**: Use `with` statement or call `close()` to avoid socket/connection leaks.
## Checkpointing (BlobCheckpointStore)
Package: `azure-eventhub-checkpointstoreblob` (sync) / `azure-eventhub-checkpointstoreblob-aio` (async)
See the [Consumer Client Retry Configuration](#consumer-client-retry-configuration) section above for a full `EventHubConsumerClient` example with `BlobCheckpointStore`.
**Common issues:**
- **Soft delete / blob versioning**: Disable both on the storage account — they cause large delays during load balancing.
- **HTTP 412/409 from storage**: Normal during partition ownership negotiation; not an error.
- **Checkpoint frequency**: Checkpoint after processing each batch, not each event, to avoid storage throttling.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |