kr7shnasomani/kairos / .agents/skills/alerting-irm
alerting-irm
VerifiedConfigure Grafana Alerting, Incident Response Management (IRM), and SLOs end-to-end — provisions Grafana-managed and data-source-managed alert rules, contact points (Slack/PagerDuty/email/webhook), notification policies with hierarchical matchers, silences, mute timings, on-call schedules and escalation chains, incident-management integrations, and SLOs with multi-window burn-rate alerts. Use when configuring alerts, debugging notification routing, setting up on-call rotations, declaring or managing incidents, defining SLOs, provisioning alerting via YAML or API, picking matchers for a notification policy, building a PagerDuty/Slack webhook receiver, or troubleshooting why an alert isn't firing — even when the user says "page me on errors", "alert me when X happens", "route this to the platform team", or "set up an SLO" without naming Alerting or IRM.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 7.6 KB |
| references/alerting.md | markdown | 4.6 KB |
| references/irm.md | markdown | 2.2 KB |
| references/slo.md | markdown | 2.1 KB |
references/slo.md
View on GitHub# SLOs and burn-rate alerts
Grafana auto-generates recording rules, dashboards, and burn-rate alerts when you define an SLO via the UI or API. Reference YAML for the generated artifacts:
## Generated recording rules
```yaml
groups:
- name: slo_availability
interval: 1m
rules:
- record: slo:availability:ratio_rate5m
expr: |
sum(rate(http_requests_total{status!~"5.."}[5m])) by (service)
/ sum(rate(http_requests_total[5m])) by (service)
- record: slo:error_budget:remaining
expr: |
(slo:availability:ratio_rate30d - 0.999) / (1 - 0.999)
```
## Burn-rate alerts
```yaml
- alert: SLOBurnRateHigh
expr: |
slo:burn_rate:ratio_rate1h > 14.4 # 1h window, 5% budget in 1h
for: 2m
labels:
severity: critical
annotations:
summary: "SLO burn rate critical for {{ $labels.service }}"
```
## Multi-window burn-rate alerts (recommended)
The single 1h-window check above fires on transient noise. Better to require burn rate above threshold in BOTH a long window (1h, the "sustained" signal) AND a short window (5m, the "still burning right now" signal):
```yaml
- alert: SLOBurnRateFast
expr: |
slo:burn_rate:ratio_rate1h > 14.4
and
slo:burn_rate:ratio_rate5m > 14.4
for: 2m
labels:
severity: critical
```
The `and` filters out: (a) a single 5m spike that wasn't actually a sustained issue (1h window stays low), and (b) a slow burn that already recovered (5m window now low). This is the Google SRE "multi-window, multi-burn-rate" pattern — see [SRE Workbook ch. 5](https://sre.google/workbook/alerting-on-slos/).
## Validating SLO config
After creating an SLO:
1. Check recording rules are running: `GET /api/v1/rules?type=record` should include the SLO's recording rules
2. Wait for the rules to evaluate at least once (default interval 1m)
3. Verify burn rate is computed: `GET /api/v1/query?query=slo:burn_rate:ratio_rate1h` should return a numeric value (not empty)
4. Test the alert: force a failure (e.g. inject errors in staging) and confirm the alert fires within the `for` duration
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| receiving-code-reviewobra/superpowersUse when receiving code review feedback, before implementing suggestions, especially if feedback seems unclear or technically questionable - requires technical rigor and verification, not performative agreement or blind implementation | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| requesting-code-reviewobra/superpowersUse when completing tasks, implementing major features, or before merging to verify work meets requirements | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| using-git-worktreesobra/superpowersUse when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallback | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| verification-before-completionobra/superpowersUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |