monte-carlo-data/mc-agent-toolkit / skills/tune-monitor
tune-monitor
VerifiedTune noisy Monte Carlo monitors, or find which ones to tune. Starts from Monte Carlo's tuning agent (stored suggestions or a fresh preview), applies only what the user approves. Metric, SQL, validation, table and agent monitors.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 22.7 KB |
| references/agent-evaluation-monitor.md | markdown | 6.9 KB |
| references/agent-metric-monitor.md | markdown | 7.5 KB |
| references/agent-trajectory-monitor.md | markdown | 6.0 KB |
| references/agent-validation-monitor.md | markdown | 6.2 KB |
| references/custom-sql-monitor.md | markdown | 5.0 KB |
| references/metric-monitor.md | markdown | 5.4 KB |
| references/table-monitor.md | markdown | 6.9 KB |
| references/validation-monitor.md | markdown | 5.6 KB |
references/validation-monitor.md
View on GitHub# Tuning Validation Monitors
This reference covers type-specific tuning guidance for validation monitors. Read this file after
determining the monitor type in Phase 1.5.
## Config fields to extract
Extract these from the `get_monitors` config response for your Phase 2 analysis:
- Alert condition tree (`alert_condition`) — a FilterGroup with UNARY, BINARY, SQL, and/or
nested GROUP nodes
- Table being validated
- Schedule interval
- Whether conditions use simple predicates or SQL expressions
---
## Key constraint: limited incident detail
Validation incidents only report **invalid row counts** — not what the rows contained. This
fundamentally limits what you can recommend without troubleshooting analysis (TSA).
**Without TSA:** You cannot determine _why_ rows are invalid or whether the alert condition
itself is too broad. Only schedule changes are safe to recommend.
**With TSA:** If troubleshooting analysis identifies specific values, patterns, or root causes
for the invalid rows, you can recommend alert condition modifications.
---
## Schedule tuning (always safe)
- If the monitor fires repeatedly for the same underlying issue (e.g., the same batch of invalid
rows detected on every run) → increase the schedule interval to reduce duplicate alerts.
- If invalid rows appear only after specific ETL jobs → align the schedule to run after those
jobs complete.
---
## Alert condition modifications (requires troubleshooting analysis)
**IMPORTANT:** Do NOT recommend alert condition changes unless TSA is present and identifies the
root cause of the invalid rows. Without knowing _what_ the invalid data looks like, condition
changes risk masking real issues.
The alert condition is a FilterGroup tree. When tuning:
### Add exclusions
Add SQL conditions or additional predicates to exclude known-valid edge cases that trigger
false positives:
```json
{
"type": "GROUP",
"operator": "AND",
"conditions": [
// ... existing conditions ...
{
"type": "SQL",
"sql": "category != 'legacy_import'"
}
]
}
```
### Tighten or loosen existing predicates
- If a BINARY condition threshold is too tight (e.g., `greater_than 0` but 1-3 invalid rows
is normal) → loosen the threshold based on observed values from TSA.
- If a UNARY null check fires on a column that is legitimately nullable for certain record
types → add an exclusion condition rather than removing the null check.
### Produce the full FilterGroup tree
When recommending changes, always output the **complete** `alert_condition` tree — not just the
modified node. The tool replaces the full condition, not individual nodes.
**NEVER** simplify or restructure the condition tree beyond the targeted change. Preserve the
existing structure and only modify what's needed.
---
## What NOT to recommend without TSA
- Removing conditions from the alert condition tree
- Changing predicate logic (e.g., `null` to `not_null`, `in_set` to different values)
- Adding new conditions based on speculation about what the invalid data might look like
If TSA is absent and the monitor is noisy, say so explicitly:
> This validation monitor is firing frequently, but without troubleshooting analysis I cannot
> determine what the invalid rows contain. I can only recommend schedule changes. To enable
> deeper tuning, run troubleshooting on recent incidents to identify the root cause.
---
## Applying changes
Use `create_or_update_validation_monitor` to update the monitor in place.
1. **Always pass `monitor_uuid=<uuid>`** so the tool updates the existing monitor rather than
creating a new one. Use the monitor UUID from Phase 1.
2. **Always dry-run first** (`dry_run=True`, the default) — show the user the YAML preview
returned in `result.yaml`, including the full updated `alert_condition` tree, and ask for
confirmation before applying.
3. **On confirmation**, call again with `dry_run=False` (and the same `monitor_uuid` plus the
same other parameters). The response carries the monitor's UUID in `result.monitor_uuid` and
a deep link in `result.instructions` — surface that to the user. `result.yaml` is `None` on
the live call by design.
4. **Stale-uuid handling.** If the monitor was deleted between read and write, the tool raises a
clear error instructing you to retry without `monitor_uuid` (turning the intent from "update"
into "create"). Confirm with the user before recreating.
### Common mistakes
- **NEVER** omit `monitor_uuid` — this creates a duplicate monitor instead of updating.
- **NEVER** apply changes without showing the dry-run preview first.
- **CRITICAL: PUT semantics.** `create_or_update_validation_monitor` with `monitor_uuid` fully
replaces the monitor's configuration — fields you omit revert to tool defaults, they are NOT
left untouched. The full config from Phase 1's `get_monitors(monitor_ids=[<uuid>],
include_fields=["config"])` call is your source of truth: re-pass every field you want to
keep (the full alert_condition tree, schedule, table, audiences, notes, priority, tags, etc.)
alongside the ones you're changing.
- **Diff the preview against the original.** Before running `dry_run=False`, compare the
rendered YAML returned in `result.yaml` against the original config — if anything you meant to
preserve is missing or changed, fix the call before committing.
- **CRITICAL:** The `alert_condition` must be a dict (JSON object), never a JSON-encoded string.
- **IMPORTANT:** Always produce the full `alert_condition` tree, not just the changed node.
The tool replaces the entire condition tree.
- **NEVER** recommend condition changes without TSA evidence — schedule changes are the only
safe lever without it.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| deprecation-and-migrationaddyosmani/agent-skillsManages deprecation and migration. Use when removing old systems, APIs, or features. Use when migrating users from one implementation to another. Use when migrating a database schema in production, such as renaming or dropping a column without downtime (expand/contract). Use when deciding whether to maintain or sunset existing code. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| performance-optimizationaddyosmani/agent-skillsOptimizes application performance across frontend, backend, queries, and databases. Use when performance requirements exist, when you suspect performance regressions, when Core Web Vitals or load times need improvement, when N+1 query patterns need fixing, or when profiling reveals bottlenecks. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| agent-cost-reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |
| agent-cost-reportthedotmack/claude-memBelievable agent cost report for any period, default the last 7 full days PT, not counting today. Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus report.json, line-items.csv, evidence.json. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |