kr7shnasomani/kairos / .agents/skills/alerting-irm
alerting-irm
VerifiedConfigure Grafana Alerting, Incident Response Management (IRM), and SLOs end-to-end — provisions Grafana-managed and data-source-managed alert rules, contact points (Slack/PagerDuty/email/webhook), notification policies with hierarchical matchers, silences, mute timings, on-call schedules and escalation chains, incident-management integrations, and SLOs with multi-window burn-rate alerts. Use when configuring alerts, debugging notification routing, setting up on-call rotations, declaring or managing incidents, defining SLOs, provisioning alerting via YAML or API, picking matchers for a notification policy, building a PagerDuty/Slack webhook receiver, or troubleshooting why an alert isn't firing — even when the user says "page me on errors", "alert me when X happens", "route this to the platform team", or "set up an SLO" without naming Alerting or IRM.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 7.6 KB |
| references/alerting.md | markdown | 4.6 KB |
| references/irm.md | markdown | 2.2 KB |
| references/slo.md | markdown | 2.1 KB |
references/irm.md
View on GitHub# IRM — on-call and incident management
## Capabilities
- **On-call schedules**: rotating shifts, overrides, escalation policies
- **Alert routing**: receive from Grafana Alerting, Prometheus, Datadog, PagerDuty, etc.
- **Incident management**: declare incidents, add participants, track tasks and timeline
- **Escalation chains**: auto-escalate if no response after N minutes
- **Integrations**: Slack, Teams, Telegram, GitHub, Jira, StatusPage
## Integration sources
| Source | Setup |
|--------|-------|
| Grafana Alerting | Native — configure in contact points (use IRM webhook receiver type) |
| Prometheus Alertmanager | Webhook URL from IRM |
| Datadog | Webhook integration |
| PagerDuty | Event integration |
| Jira | Issue-triggered alerts |
| Custom | Generic webhook |
## Routing from Grafana Alerting to IRM
In `provisioning/alerting/contact_points.yaml`:
```yaml
apiVersion: 1
contactPoints:
- orgId: 1
name: irm-routing
receivers:
- uid: irm-webhook
type: webhook
settings:
url: https://<region>.grafana.net/api/plugins/grafana-irm-app/resources/integrations/<integration-id>/webhook
httpMethod: POST
```
Then route alerts to this contact point via a notification policy matcher (e.g. `severity = page`).
## Verifying the IRM integration
1. Trigger a test alert in Grafana (use the "Test" button in the alert rule UI, or send a fake one via `POST /api/v1/alerts`)
2. In the IRM UI, the alert should appear under the matching integration within ~30 seconds
3. If it doesn't:
- Check the webhook URL matches the integration's URL in IRM
- Check the notification policy actually routes to the `irm-routing` contact point
- Check Grafana logs for HTTP errors on the webhook POST
## Common failure modes
| Symptom | Likely cause |
|---|---|
| Alert fires in Grafana but never reaches IRM | Notification policy doesn't match the alert's labels; check matchers in `policies.yaml` |
| Alert reaches IRM but no on-call notification | Escalation chain misconfigured (e.g. no users in the rotation, or quiet hours suppressing) |
| Same alert opens duplicate IRM incidents | Group_by on the notification policy isn't including the unique-ifying labels |
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| receiving-code-reviewobra/superpowersUse when receiving code review feedback, before implementing suggestions, especially if feedback seems unclear or technically questionable - requires technical rigor and verification, not performative agreement or blind implementation | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| requesting-code-reviewobra/superpowersUse when completing tasks, implementing major features, or before merging to verify work meets requirements | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| using-git-worktreesobra/superpowersUse when starting feature work that needs isolation from current workspace or before executing implementation plans - ensures an isolated workspace exists via native tools or git worktree fallback | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| verification-before-completionobra/superpowersUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |