Skip to content

meltedinhex/analyst-ai-pack / skills/tuning-detections-to-reduce-false-positives

tuning-detections-to-reduce-false-positives

Verified

Tunes noisy detections by analyzing historical alert data to quantify false-positive sources, recommend allow-list filters and threshold adjustments, and measure precision impact. Activates for requests to tune a detection, reduce false positives, or analyze alert noise and recommend filters.

FileKindSize
LICENSEother340 B
SKILL.mdskill md2.7 KB
references/api-reference.mdmarkdown1.3 KB
scripts/analyst.pyscript3.1 KB

references/api-reference.md

View on GitHub
# API Reference: Tuning Detections to Reduce False Positives

## Overview

`scripts/analyst.py` analyzes labeled alert history to compute current precision and rank a field's
values by false-positive contribution, recommending allow-list filters for values with zero
true-positive loss and projecting the precision gain. It parses alert data only and executes
nothing.

## Dependencies

| Package | Version | Purpose |
|---------|---------|---------|
| (stdlib) | — | csv, json, file IO |

## Core functions

### `analyze(path, label_field, field)`
Returns current tp/fp/precision, FP contributors, recommended safe filters, and projected precision.

## Label values

True positive: `tp`/`true_positive`/`malicious`/`1`. False positive: `fp`/`false_positive`/
`benign`/`0`.

## Safety

Only values with **zero** true positives are recommended as filters, so tuning never suppresses
known true positives.

## Usage

```bash
python scripts/analyst.py analyze alerts.csv --label verdict --field ParentImage
```

## Sources

- Sigma rule specification (filter/condition) — https://github.com/SigmaHQ/sigma-specification
- Precision/false-positive-rate detection-engineering concepts — https://attack.mitre.org/resources/
- FP-contribution ranking and safe-filter approach from detection-engineering practice
slack-gif-creatoranthropics/skillsKnowledge and utilities for creating animated GIFs optimized for Slack. Provides constraints, validation tools, and animation concepts. Use when users request animated GIFs for Slack like "make me a GIF of X doing Y for Slack."Stars: 198kUpdated Claude CodeCodexVerifiedInstall
ponytailDietrichGebert/ponytailLazy senior dev mode for any coding task: the smallest change that fully solves it, and a reply a busy human understands in one read.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailHarvest every shortcut: comment into one debt ledger, so deferrals get tracked instead of forgotten. One-shot report.Stars: 194kUpdated Claude CodeCodexVerifiedInstall
ponytail-debtDietrichGebert/ponytailList every `shortcut:` comment (and older `ponytail:` ones) as a debt ledger. One-shot report, changes nothing. Use for "ponytail debt", "what did ponytail defer", "list the shortcuts", /ponytail-debt.Stars: 194kUpdated Claude CodeCodexVerifiedInstall