Skip to content

meltedinhex/analyst-ai-pack / skills/validating-detections-with-atomic-tests

validating-detections-with-atomic-tests

Verified

Validates detection coverage by mapping Atomic Red Team tests to ATT&CK techniques, checking which of your detections cover each tested technique, and reporting coverage gaps. Activates for requests to validate detections with atomic tests, map Atomic Red Team to coverage, or find detection gaps against tested techniques.

Validating Detections With Atomic Tests

When to Use

  • You want to validate that your detections fire for specific ATT&CK techniques by correlating Atomic Red Team test definitions with your detection inventory.
  • You are measuring coverage and identifying gaps before or after running atomics.

Do not use the atomics themselves on production systems without authorization and isolation — they execute adversary behaviors. This skill performs mapping/coverage analysis offline.

Prerequisites

  • An export of Atomic test→technique mappings (CSV/JSON) and your detection inventory mapped to ATT&CK techniques.

Workflow

Step 1: Compute coverage

python scripts/analyst.py coverage --atomics atomics.json --detections detections.json

Joins atomic-tested techniques with your detections' covered techniques and reports covered vs uncovered techniques.

Step 2: Prioritize gaps

Rank uncovered techniques by the number of atomic tests (more atomics → more validated adversary behavior you currently miss).

Step 3: Validate live (authorized/isolated)

For covered techniques, run the corresponding atomic in an isolated range and confirm the detection fires; record results.

Step 4: Close gaps

Author detections for high-priority uncovered techniques and re-run coverage.

Validation

  • Coverage join keys on ATT&CK technique IDs (including sub-techniques).
  • Covered and uncovered technique lists are complete and disjoint.
  • Gap prioritization reflects atomic-test counts.

Pitfalls

  • Sub-technique vs parent-technique mismatch (T1059 vs T1059.001) skewing coverage.
  • Claiming coverage from a rule that exists but is disabled/untuned.
  • Running atomics outside an isolated, authorized range.

References

finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the workStars: 319kUpdated Claude CodeCodexVerifiedInstall
systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesStars: 319kUpdated Claude CodeCodexVerifiedInstall
test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation codeStars: 319kUpdated Claude CodeCodexVerifiedInstall
skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy.Stars: 196kUpdated Claude CodeCodexVerifiedInstall