monte-carlo-data/mc-agent-toolkit / skills/tune-monitor
tune-monitor
VerifiedTune noisy Monte Carlo monitors, or find which ones to tune. Starts from Monte Carlo's tuning agent (stored suggestions or a fresh preview), applies only what the user approves. Metric, SQL, validation, table and agent monitors.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 22.7 KB |
| references/agent-evaluation-monitor.md | markdown | 6.9 KB |
| references/agent-metric-monitor.md | markdown | 7.5 KB |
| references/agent-trajectory-monitor.md | markdown | 6.0 KB |
| references/agent-validation-monitor.md | markdown | 6.2 KB |
| references/custom-sql-monitor.md | markdown | 5.0 KB |
| references/metric-monitor.md | markdown | 5.4 KB |
| references/table-monitor.md | markdown | 6.9 KB |
| references/validation-monitor.md | markdown | 5.6 KB |
references/custom-sql-monitor.md
View on GitHub# Tuning Custom SQL Monitors
This reference covers type-specific tuning guidance for custom SQL monitors (custom rules).
Read this file after determining the monitor type in Phase 1.5.
## Config fields to extract
Extract these from the `get_monitors` config response for your Phase 2 analysis:
- SQL query text (`sql` or `custom_sql`)
- Alert conditions — each has an `operator` and either a `threshold_value` (explicit) or ML
threshold configuration
- Warehouse name
- Schedule interval
**IMPORTANT:** Determine whether the monitor uses **ML thresholds** or **explicit thresholds**.
This affects which tuning levers are available. ML-threshold monitors support sensitivity tuning
(covered in Phase 3 of SKILL.md). Explicit-threshold monitors require direct threshold adjustment
(covered below).
---
## Threshold adjustment (explicit thresholds)
For monitors with explicit thresholds (`GT`, `LT`, `GTE`, `LTE`, `EQ`, `NE`):
**When to recommend loosening a threshold:**
- Anomalies are consistently marginal — the observed value just barely crosses the threshold
- The margin between observed and threshold is small relative to the metric's natural variance
- Most alerts are assessed as normal variation, not genuine issues
**CRITICAL:** Always explain what the threshold value represents in business terms before
recommending a change. The user needs to understand what "changing GT 0 to GT 5" means for their
data quality.
**Common pattern:** If the query returns a count of "bad" rows and the threshold is 0, but normal
operations produce 1-3 rows that match the condition (e.g., stale records, delayed updates),
recommend raising the threshold to a reasonable floor based on observed values.
**Examples:**
```yaml
# Before: fires on any bad row
alert_conditions:
- operator: GT
threshold_value: 0
# After: tolerates up to 5 (based on observed noise floor of 1-3)
alert_conditions:
- operator: GT
threshold_value: 5
```
**NEVER** recommend a threshold change without citing the observed anomaly values from the report.
---
## SQL query modifications
When the SQL itself contributes to noise, recommend targeted modifications:
**When to recommend SQL changes:**
- The query lacks time-window filters and picks up stale data
- Certain dimensions or categories are known-noisy and should be excluded
- NULL handling is missing and causes spurious results
- The query could benefit from a more targeted WHERE clause
**NEVER** rewrite the full SQL without showing a diff. Present the original query and the proposed
change side by side so the user can review exactly what changed.
**IMPORTANT:** SQL syntax varies by warehouse. When suggesting modifications, match the warehouse
dialect:
| Warehouse | Date arithmetic example |
|-----------|------------------------|
| Snowflake | `DATEADD('day', -7, CURRENT_TIMESTAMP())` |
| BigQuery | `TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY)` |
| Redshift | `DATEADD(day, -7, GETDATE())` |
| Databricks | `DATE_SUB(CURRENT_TIMESTAMP(), 7)` |
---
## Applying changes
Use `create_or_update_sql_monitor` to update the monitor in place.
1. **Always pass `monitor_uuid=<uuid>`** so the tool updates the existing monitor rather than
creating a new one. Use the monitor UUID from Phase 1.
2. **Always dry-run first** (`dry_run=True`, the default) — show the user the YAML preview
returned in `result.yaml` and ask for confirmation before applying.
3. **On confirmation**, call again with `dry_run=False` (and the same `monitor_uuid` plus the
same other parameters). The response carries the monitor's UUID in `result.monitor_uuid` and
a deep link in `result.instructions` — surface that to the user. `result.yaml` is `None` on
the live call by design.
4. **Stale-uuid handling.** If the monitor was deleted between read and write, the tool raises a
clear error instructing you to retry without `monitor_uuid` (turning the intent from "update"
into "create"). Confirm with the user before recreating.
### Common mistakes
- **NEVER** omit `monitor_uuid` — this creates a duplicate monitor instead of updating.
- **NEVER** apply changes without showing the dry-run preview first.
- **CRITICAL: PUT semantics.** `create_or_update_sql_monitor` with `monitor_uuid` fully replaces
the monitor's configuration — fields you omit revert to tool defaults, they are NOT left
untouched. The full config from Phase 1's `get_monitors(monitor_ids=[<uuid>],
include_fields=["config"])` call is your source of truth: re-pass every field you want to
keep (sql, all alert_conditions, schedule, warehouse, audiences, notes, priority, tags, etc.)
alongside the ones you're changing.
- **Diff the preview against the original.** Before running `dry_run=False`, compare the
rendered YAML returned in `result.yaml` against the original config — if anything you meant to
preserve is missing or changed, fix the call before committing.
- **CRITICAL:** When modifying the SQL query, ensure the query still returns a single numeric
value. A query that returns multiple rows or non-numeric data will break the monitor.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| deprecation-and-migrationaddyosmani/agent-skillsManages deprecation and migration. Use when removing old systems, APIs, or features. Use when migrating users from one implementation to another. Use when migrating a database schema in production, such as renaming or dropping a column without downtime (expand/contract). Use when deciding whether to maintain or sunset existing code. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| performance-optimizationaddyosmani/agent-skillsOptimizes application performance across frontend, backend, queries, and databases. Use when performance requirements exist, when you suspect performance regressions, when Core Web Vitals or load times need improvement, when N+1 query patterns need fixing, or when profiling reveals bottlenecks. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| agent-cost-reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |
| agent-cost-reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |