borghei/Claude-Skills / engineering/observability-designer
observability-designer
VerifiedDesign observability strategies: SLI/SLO frameworks, alerting, and dashboards. Use when instrumenting a production service, tuning alert rules, designing Grafana dashboards, defining SLOs and error budgets, or reducing alert fatigue.
| File | Kind | Size |
|---|---|---|
| README.md | markdown | 11.6 KB |
| SKILL.md | skill md | 7.5 KB |
| assets/sample_alerts.json | config | 8.0 KB |
| assets/sample_service_api.json | config | 1.7 KB |
| assets/sample_service_web.json | config | 2.4 KB |
| expected_outputs/sample_dashboard.json | config | 19.1 KB |
| expected_outputs/sample_slo_framework.json | config | 21.6 KB |
| references/alert_design_patterns.md | markdown | 11.7 KB |
| references/dashboard_best_practices.md | markdown | 13.8 KB |
| references/dashboards-logs-traces.md | markdown | 3.4 KB |
| references/slo-and-alerting.md | markdown | 4.1 KB |
| references/slo_cookbook.md | markdown | 10.4 KB |
| references/tools-integration-and-troubleshooting.md | markdown | 8.9 KB |
| scripts/alert_optimizer.py | script | 45.6 KB |
| scripts/dashboard_generator.py | script | 48.4 KB |
| scripts/slo_designer.py | script | 28.8 KB |
references/dashboards-logs-traces.md
View on GitHub# Dashboards, Logs, Traces & a Worked Example
Read this when building dashboards, defining structured logs, choosing trace sampling, authoring runbooks, or you want a complete worked observability spec plus cost-optimization guidance.
## Dashboard Design Rules
- **Hierarchy**: Overview (all services) > Service (one service) > Component (e.g., database) > Instance
- **Panel limit**: Maximum 7 panels per screen to manage cognitive load
- **Reference lines**: Always show SLO targets and capacity thresholds
- **Time defaults**: 4 hours for incident investigation, 7 days for trend analysis
- **Role-based views**: SRE (operational), Developer (debug), Executive (reliability summary)
## Structured Log Format
```json
{
"timestamp": "2025-11-05T14:30:00Z",
"level": "ERROR",
"service": "payment-api",
"trace_id": "abc123def456",
"span_id": "789ghi",
"message": "Payment processing failed",
"error_code": "PAYMENT_TIMEOUT",
"duration_ms": 5023,
"customer_id": "cust_42",
"environment": "production"
}
```
**Log levels**: DEBUG (local dev only), INFO (request lifecycle), WARN (degraded but functional), ERROR (failed operation), FATAL (service cannot continue).
## Trace Sampling Strategies
| Strategy | When to Use | Trade-off |
|----------|------------|-----------|
| Head-based (10%) | Development, low-traffic services | Misses rare errors |
| Tail-based | Production, high-traffic | Captures errors/slow requests; higher resource cost |
| Adaptive | Variable traffic patterns | Adjusts rate based on load; more complex to configure |
## Runbook Template
```markdown
# Alert: [Alert Name]
## What It Means
[One sentence explaining the alert condition]
## Impact
[User-facing vs internal; affected services]
## Investigation Steps
1. Check dashboard: [link] (1 min)
2. Review recent deploys: [link] (2 min)
3. Check dependent services: [list] (2 min)
4. Review logs: [query] (3 min)
## Resolution Actions
- If [condition A]: [action]
- If [condition B]: [action]
- If unclear: Escalate to [team] via [channel]
## Post-Incident
- [ ] Update incident timeline
- [ ] File post-mortem if > 5 min user impact
```
## Example: E-Commerce Payment Service Observability
```yaml
service: payment-api
tier: T1 (revenue-critical)
owner: payments-team
slis:
availability:
metric: "1 - rate(http_5xx) / rate(http_total)"
slo: 99.95%
error_budget: 21.6 min/month
latency_p99:
metric: "histogram_quantile(0.99, http_duration_seconds)"
slo: < 800ms
error_rate:
metric: "rate(payment_failures) / rate(payment_attempts)"
slo: < 0.5%
alerts:
- name: PaymentHighErrorRate
expr: "rate(payment_failures[5m]) / rate(payment_attempts[5m]) > 0.01"
for: 2m
severity: critical
runbook: "https://wiki.internal/runbooks/payment-errors"
dashboard_panels:
- Payment success rate (gauge)
- Transaction volume (time series)
- P50/P95/P99 latency (time series)
- Error breakdown by type (stacked bar)
- Downstream dependency health (status map)
- Error budget remaining (gauge)
```
## Cost Optimization
- **Metric retention**: 15-day full resolution, 90-day downsampled, 1-year aggregated
- **Log sampling**: Sample DEBUG/INFO at 10% in high-throughput services; always keep ERROR/FATAL at 100%
- **Trace sampling**: Tail-based sampling retains only errors and slow requests (> P99)
- **Cardinality management**: Alert on any metric with > 10K unique label combinations
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| brainstormingobra/superpowersYou MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation). | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| brand-guidelinesanthropics/skillsApplies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| frontend-designanthropics/skillsGuidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| mcp-builderanthropics/skillsGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK). | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |