Skip to content

anyshift-io/sre-skills / skills/sqs-queue-auditor

sqs-queue-auditor

Verified

FileKindSize
FAILURE_MODES.mdmarkdown6.6 KB
README.mdmarkdown4.9 KB
SKILL.mdskill md13.3 KB
examples/01-no-dlq.mdmarkdown2.9 KB
examples/02-dlq-retention-shorter-than-source.mdmarkdown3.3 KB
examples/03-maxreceivecount-too-low.mdmarkdown2.8 KB
examples/04-poison-ages-out-before-dlq.mdmarkdown3.9 KB
examples/05-default-visibility-short-retention.mdmarkdown3.2 KB
examples/06-public-queue-policy.mdmarkdown3.6 KB
examples/07-fifo-dedup-off.mdmarkdown2.8 KB
examples/08-clean-standard.mdmarkdown2.6 KB
fixtures/01-no-dlq/queue.jsonconfig564 B
fixtures/02-dlq-retention-shorter-than-source/dlq.jsonconfig649 B
fixtures/02-dlq-retention-shorter-than-source/queue.jsonconfig632 B
fixtures/03-maxreceivecount-too-low/dlq.jsonconfig659 B
fixtures/03-maxreceivecount-too-low/queue.jsonconfig637 B
fixtures/04-poison-ages-out-before-dlq/dlq.jsonconfig663 B
fixtures/04-poison-ages-out-before-dlq/queue.jsonconfig648 B
fixtures/05-default-visibility-short-retention/dlq.jsonconfig650 B
fixtures/05-default-visibility-short-retention/queue.jsonconfig631 B
fixtures/06-public-queue-policy/dlq.jsonconfig663 B
fixtures/06-public-queue-policy/queue.jsonconfig908 B
fixtures/07-fifo-dedup-off/dlq.jsonconfig748 B
fixtures/07-fifo-dedup-off/queue.jsonconfig804 B
fixtures/08-clean-standard/dlq.jsonconfig672 B
fixtures/08-clean-standard/queue.jsonconfig1.0 KB
tests/README.mdmarkdown4.2 KB
tests/_audit.pyscript21.8 KB
tests/_replay.pyscript720 B
tests/eval/README.mdmarkdown8.4 KB
tests/eval/eval_results.jsonconfig220.4 KB
tests/eval/judge_prompt.mdmarkdown1.4 KB
tests/eval/queues.pyscript7.6 KB
tests/eval/rubric.mdmarkdown2.6 KB
tests/eval/run_eval.pyscript11.5 KB
tests/replay_01_no_dlq.pyscript1.5 KB
tests/replay_02_dlq_retention.pyscript1.5 KB
tests/replay_03_maxreceivecount.pyscript1.3 KB
tests/replay_04_poison_ages_out.pyscript1.6 KB
tests/replay_05_default_visibility.pyscript1.8 KB
tests/replay_06_public_policy.pyscript1.4 KB
tests/replay_07_fifo_dedup_off.pyscript1.4 KB
tests/replay_08_clean_standard.pyscript1.6 KB

examples/03-maxreceivecount-too-low.md

View on GitHub
# Worked example 3: maxReceiveCount too low (R2)

A dead-letter queue wired correctly, sized correctly on retention, and set to give up after a single failed delivery. Transient blips dead-letter perfectly good messages. Fixtures and replay test under `../fixtures/03-maxreceivecount-too-low/` and `../tests/replay_03_maxreceivecount.py`.

## Scenario

- **Queue**: `email-dispatch`, with DLQ `email-dispatch-dlq`.
- **Symptom**: the DLQ has over a thousand messages in it. On inspection, almost all of them are valid emails that would have sent fine on a retry. A 90-second SES throttle this morning dead-lettered every message in flight, because the queue gives up after one attempt.

## Step 1: parse the attributes

Source (`queue.json`):
```
MessageRetentionPeriod 345600   (4 days)
RedrivePolicy          {"deadLetterTargetArn":"...:email-dispatch-dlq","maxReceiveCount":1}
```
DLQ (`dlq.json`):
```
MessageRetentionPeriod 1209600  (14 days)
```

`maxReceiveCount` parses to `1`.

## Step 2: audit the redrive path

The DLQ is present and its retention (14 days) is longer than the source (4 days), so R3 does not fire. But `maxReceiveCount=1` is below the sane band, which is **R2 (medium)**. A message that fails its single delivery attempt goes straight to the DLQ. There is no retry, so any recoverable, transient downstream failure (a rolling deploy, a brief throttle, a 2-second timeout) sends a good message to dead-letter. The DLQ fills with messages that were never poison, which masks the ones that are: when something is genuinely broken, it is buried under a thousand false positives.

## Steps 3-5: lifecycle, exposure, FIFO

Clean. `1 x 60 = 60s` is far below retention (no R4). Visibility is a deliberate 60s (no R5), retention is the default (no R6), SSE is on, no resource policy, standard queue. R2 is the only finding.

## Finding

| Code | Severity | Attribute | Fix |
|---|---|---|---|
| R2 | medium | `RedrivePolicy.maxReceiveCount` | Raise `maxReceiveCount` into the 3-10 band so transient failures are retried before quarantine. |

## Boundary

The audit proves the queue dead-letters on the first failure. It cannot tell you what fraction of the DLQ is transient-failure noise versus genuine poison.

- The ratio of recoverable to poison messages in the DLQ is a property of the message contents and the consumer's failure reasons, not a queue attribute. Join: DLQ to its consumers.
- Whether a redrive of the DLQ would now succeed is the downstream's current health, not a static attribute. Join: queue to its metrics over time.

## Why this is the R2 reference

It is the opposite failure from a missing DLQ: the dead-letter path works *too eagerly*. The fix is a single integer, but the symptom (a DLQ full of valid messages, the real poison invisible inside it) looks like a content problem until you read the one attribute that explains it.
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall