cogni-dao/cogni / .claude/skills/monitoring-expert
monitoring-expert
VerifiedConfigures monitoring systems, implements structured logging pipelines, creates Prometheus/Grafana dashboards, defines alerting rules, and instruments distributed tracing. Implements Prometheus/Grafana stacks, conducts load testing, performs application profiling, and plans infrastructure capacity. Use when setting up application monitoring, adding observability to services, debugging production issues with logs/metrics/traces, running load tests with k6 or Artillery, profiling CPU/memory bottlenecks, or forecasting capacity needs.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 8.0 KB |
| references/alerting-rules.md | markdown | 3.6 KB |
| references/application-profiling.md | markdown | 6.6 KB |
| references/capacity-planning.md | markdown | 8.2 KB |
| references/dashboards.md | markdown | 3.9 KB |
| references/opentelemetry.md | markdown | 3.4 KB |
| references/performance-testing.md | markdown | 5.9 KB |
| references/prometheus-metrics.md | markdown | 3.2 KB |
| references/structured-logging.md | markdown | 3.0 KB |
references/alerting-rules.md
View on GitHub# Alerting Rules
## Prometheus Alert Rules
```yaml
# alerts.yml
groups:
- name: application
rules:
- alert: HighErrorRate
expr: |
sum(rate(http_requests_total{status=~"5.."}[5m]))
/
sum(rate(http_requests_total[5m])) > 0.05
for: 5m
labels:
severity: critical
annotations:
summary: High error rate detected
description: Error rate is {{ $value | humanizePercentage }}
- alert: HighLatency
expr: |
histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m])) > 1
for: 5m
labels:
severity: warning
annotations:
summary: High latency detected
description: 95th percentile latency is {{ $value }}s
- alert: ServiceDown
expr: up == 0
for: 1m
labels:
severity: critical
annotations:
summary: Service {{ $labels.instance }} is down
- name: infrastructure
rules:
- alert: HighMemoryUsage
expr: |
(node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes)
/ node_memory_MemTotal_bytes > 0.9
for: 5m
labels:
severity: warning
annotations:
summary: High memory usage on {{ $labels.instance }}
- alert: HighCPUUsage
expr: |
100 - (avg by(instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 80
for: 10m
labels:
severity: warning
annotations:
summary: High CPU usage on {{ $labels.instance }}
- alert: DiskSpaceLow
expr: |
(node_filesystem_avail_bytes / node_filesystem_size_bytes) < 0.1
for: 5m
labels:
severity: critical
annotations:
summary: Disk space low on {{ $labels.instance }}
```
## Alert Design Principles
```yaml
# Good alert: Actionable, specific
- alert: DatabaseConnectionPoolExhausted
expr: db_pool_available_connections == 0
for: 2m
annotations:
runbook_url: https://wiki.example.com/runbooks/db-pool
# Bad alert: Too noisy, not actionable
- alert: AnyError
expr: errors_total > 0 # Will always fire
```
## Severity Levels
| Severity | Response | Example |
| ---------- | ---------------- | ----------------------- |
| `critical` | Page immediately | Service down, data loss |
| `warning` | Investigate soon | High latency, low disk |
| `info` | Check in morning | Unusual traffic pattern |
## Alertmanager Configuration
```yaml
# alertmanager.yml
global:
slack_api_url: "https://hooks.slack.com/..."
route:
receiver: "slack-notifications"
group_by: ["alertname", "severity"]
group_wait: 30s
group_interval: 5m
repeat_interval: 4h
routes:
- match:
severity: critical
receiver: "pagerduty"
- match:
severity: warning
receiver: "slack-notifications"
receivers:
- name: "slack-notifications"
slack_configs:
- channel: "#alerts"
send_resolved: true
- name: "pagerduty"
pagerduty_configs:
- service_key: "your-key"
```
## Quick Reference
| Field | Purpose |
| ------------- | ------------------------- |
| `expr` | PromQL query |
| `for` | Duration before firing |
| `labels` | Classification (severity) |
| `annotations` | Human-readable info |
| Threshold | Use |
| ---------------- | -------- |
| Error rate > 5% | Critical |
| p95 latency > 1s | Warning |
| Disk < 10% | Critical |
| Memory > 90% | Warning |
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |