Skip to content

monte-carlo-data/mc-agent-toolkit / skills/incident-response

monte-carlo-incident-response

Verified

Orchestrate incident response — triage, root cause, remediate, prevent recurrence. USE WHEN active alerts, data broken, stale, pipeline failure, or investigate and fix a data incident.

Monte Carlo Incident Response Workflow

This workflow orchestrates the full lifecycle of a data incident by sequencing existing Monte Carlo skills. Apart from reading what Monte Carlo's agents already found, it does not contain investigation or remediation logic itself — each step loads the relevant skill's SKILL.md which has the actual instructions.

Monte Carlo tool routing (required): For Monte Carlo operations such as get_alerts, search and get_asset, use the tool ID your client exposes on the connection for the intended account. Proceed when there is one suitable connection and no evidence of an account or source mismatch. If the choice is ambiguous, ask which connection to use before making a Monte Carlo call; if you can't get an answer, leave the call pending. Use the selected connection throughout. If it is unavailable or lacks the operation, follow this skill's fallback where it has one, or report what is missing.

With the Agent Toolkit plugin: When using the toolkit plugin or its installer, prefer the connection it provides or configures (normally named monte-carlo-mcp) when it is suitable for the intended account. If you know that connection is missing, confirm the intended alternative before using another one.

When to activate this workflow

Activate when:

  • Context detection routes here (active alerts detected + incident intent)
  • User invokes /mc-incident-response
  • User asks to "respond to an incident", "handle this alert", "triage and fix"
  • User describes a data quality problem: "data is broken", "table is stale", "alert firing"

When NOT to activate this workflow

  • User wants to create monitors or check coverage without an active incident — use proactive monitoring workflow
  • User is editing a dbt model — defer to prevent skill (auto-activates via hooks)
  • User wants to check table health without an incident context — use asset-health directly
  • A skill is already active and handling the user's request

Workflow Steps

Step 0: Read what is already known — direct reads, no agent runs
Step 1 (conditional): Triage — when user has multiple/unknown alerts
Step 2: Root Cause Analysis — the core investigation
Step 3: Remediation — fix or escalate
Step 4 (optional): Prevent Recurrence — add monitoring

Step 0: Read what is already known

Once you know which alert, always read what Monte Carlo's agents already found, even if the alert fired minutes ago. Do this before choosing an entry point:

  1. The triage verdict, from that alert's get_alerts fields triage_priority and triage_result (no extra call).
  2. get_troubleshooting_agent_results(incident_id=...): an existing root cause.
  3. If that succeeded, get_remediation_instructions(alert_id=...): the recommended fix.

Do not call triage_alert or run_troubleshooting_agent before these reads. Full order, empty-result handling, tuning and teammate activity: references/prior-agent-work.md.

Report what you found and what is missing. Then run nothing unasked: go on to Step 2 or 3 only when the user asked to investigate or fix.

  • A root cause and plan exist → report them and offer Step 3: Remediation.
  • A root cause exists, with no plan → report it and offer Step 3.
  • An analysis is running → report it and poll get_troubleshooting_agent_results. Do not start Step 2's troubleshooting run.
  • Nothing exists yet → say so. If the user asked to investigate, continue with the entry point below; otherwise offer it.

Determine entry point

Before starting, determine which step to enter based on the user's context:

  • User has no specific alert ("I have alerts firing", "what's going on?") → Start at Step 1: Triage
  • User has a specific alert ID or table ("alert ABC-123", "stg_payments is stale") → Skip to Step 2: Root Cause Analysis
  • User knows the root cause ("the ETL job failed, help me fix it") → Skip to Step 3: Remediation
  • Alert is an agent-monitor alert (alert_types starting with "Agent ", or the user's issue is about an AI agent) → for the investigation, read ../troubleshoot-agent-traces/SKILL.md instead of ../analyze-root-cause/SKILL.md; the remediation and monitoring steps still apply
  • Ambiguous → Ask: "Do you have a specific alert or table you want to investigate, or should I check your recent alerts first?"

Step 1: Triage (conditional)

Skill: Read and follow ../automated-triage/SKILL.md

Goal: Fetch recent alerts, score them by confidence and impact, identify which ones need investigation.

When to run: Only when the user doesn't already have a specific alert or incident to investigate. This step helps narrow down "I have alerts" into "these specific alerts need attention."

Scope MCP calls tightly. On large accounts, broad queries return hundreds of results, overflow the tool-result token limit, spill to disk, and force chunk reads — burning user tokens and exhausting the turn budget. Minimum scoping for tools this workflow touches:

  • get_alerts → time filter (created_after, default last 7 days) + at least one of warehouse, table_names, severity
  • search → needed to resolve a table name to its MCON (get_asset is most reliable with an MCON). Always pass limit (e.g. 5), the table name as query, and filter by warehouse_uuid or database/schema. warehouse_types alone is too broad. If multiple matches return: (1) auto-pick the match whose warehouse_display_name matches the user's named warehouse — do NOT stop to ask; (2) failing that, prefer the is_key_asset: true match; (3) only ask the user when none of these resolve it
  • get_monitors → filter by mcons or warehouse_uuid

If scope is missing, ask the user before calling: "Which warehouse?", "How far back — today, this week?", "Any specific severity?".

Transition to Step 2: Once high-priority alert(s) are identified, tell the user:

"I've identified [N] high-priority alerts. Let me investigate the root cause of [specific alert/table]. Moving to root cause analysis."

Then proceed to Step 2 with the identified alert context.


Step 2: Root Cause Analysis

Skill: Read and follow ../analyze-root-cause/SKILL.md

Goal: Investigate why the issue occurred — trace lineage, check ETL changes, analyze query modifications, profile data.

This is the core step. Most workflow entries start here.

Investigate linearly — do not re-call tools. Walk through the investigation once: (1) find the table, (2) fetch its alerts and freshness, (3) check lineage, (4) check recent queries/ETL. Call each tool at most once per table. If a tool result is insufficient, move to the next signal rather than re-calling with different params — burning turns on redundant calls exhausts the budget before the root cause is reached.

Transition to Step 3: When the root cause is identified (or the investigation reaches its limit), summarize findings and tell the user:

"Root cause identified: [summary]. Would you like me to help remediate this, or is the investigation sufficient?"

If the user wants to proceed, move to Step 3. If they say "that's enough", stop.


Step 3: Remediation

Skill: Read and follow ../remediation/SKILL.md

Goal: Fix the issue using available tools, or escalate with full context if the fix requires actions outside the agent's capability.

Transition to Step 4: After remediation is complete (fix applied or escalation documented), offer prevention:

"The issue has been [fixed/escalated]. The root cause was [X]. Want me to help add a monitor to detect this type of issue earlier next time?"

If the user says yes, move to Step 4. If no, the workflow is complete.


Step 4: Prevent Recurrence (optional)

Skill: Read and follow ../monitoring-advisor/SKILL.md

When loading monitoring-advisor for this step, frame the request as direct monitor creation — not coverage analysis. The user already knows what they want to monitor (the thing that just broke). Example framing:

"Based on the incident, I recommend adding a [freshness/volume/validation] monitor on [table]. Let me create the monitor configuration."

Goal: Add or update a monitor to catch this class of issue in the future.

Do not force this step. It is optional — offer it after remediation, and respect if the user declines.


Orchestration Rules

  • Users can enter at any step. The entry point section above determines where to start.
  • Each step loads the actual skill's SKILL.md via relative path. Apart from Step 0's reads of existing agent results, this workflow does not replicate skill logic — it sequences it.
  • Context carries forward through conversation naturally. Alert IDs, table names, root cause findings from earlier steps are available to later steps without explicit state passing.
  • No state tracking or hooks. This is purely prompt-driven sequencing.
  • User can exit anytime. If they say "that's enough" or "stop", respect it immediately.
  • Do not skip back. The workflow moves forward. If the user wants to re-investigate after remediation, they can start a new workflow or invoke a skill directly.
writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 197kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall