meltedinhex/analyst-ai-pack / skills/validating-detections-with-atomic-tests
validating-detections-with-atomic-tests
VerifiedValidates detection coverage by mapping Atomic Red Team tests to ATT&CK techniques, checking which of your detections cover each tested technique, and reporting coverage gaps. Activates for requests to validate detections with atomic tests, map Atomic Red Team to coverage, or find detection gaps against tested techniques.
Validating Detections With Atomic Tests
When to Use
- You want to validate that your detections fire for specific ATT&CK techniques by correlating Atomic Red Team test definitions with your detection inventory.
- You are measuring coverage and identifying gaps before or after running atomics.
Do not use the atomics themselves on production systems without authorization and isolation — they execute adversary behaviors. This skill performs mapping/coverage analysis offline.
Prerequisites
- An export of Atomic test→technique mappings (CSV/JSON) and your detection inventory mapped to ATT&CK techniques.
Workflow
Step 1: Compute coverage
python scripts/analyst.py coverage --atomics atomics.json --detections detections.json
Joins atomic-tested techniques with your detections' covered techniques and reports covered vs uncovered techniques.
Step 2: Prioritize gaps
Rank uncovered techniques by the number of atomic tests (more atomics → more validated adversary behavior you currently miss).
Step 3: Validate live (authorized/isolated)
For covered techniques, run the corresponding atomic in an isolated range and confirm the detection fires; record results.
Step 4: Close gaps
Author detections for high-priority uncovered techniques and re-run coverage.
Validation
- Coverage join keys on ATT&CK technique IDs (including sub-techniques).
- Covered and uncovered technique lists are complete and disjoint.
- Gap prioritization reflects atomic-test counts.
Pitfalls
- Sub-technique vs parent-technique mismatch (T1059 vs T1059.001) skewing coverage.
- Claiming coverage from a rule that exists but is disabled/untuned.
- Running atomics outside an isolated, authorized range.
References
- See
references/api-reference.mdfor the coverage tool. - Atomic Red Team and ATT&CK references (linked in frontmatter).
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |