Skip to content

anyshift-io/sre-skills / skills/sqs-queue-auditor

sqs-queue-auditor

Verified

FileKindSize
FAILURE_MODES.mdmarkdown6.6 KB
README.mdmarkdown4.9 KB
SKILL.mdskill md13.3 KB
examples/01-no-dlq.mdmarkdown2.9 KB
examples/02-dlq-retention-shorter-than-source.mdmarkdown3.3 KB
examples/03-maxreceivecount-too-low.mdmarkdown2.8 KB
examples/04-poison-ages-out-before-dlq.mdmarkdown3.9 KB
examples/05-default-visibility-short-retention.mdmarkdown3.2 KB
examples/06-public-queue-policy.mdmarkdown3.6 KB
examples/07-fifo-dedup-off.mdmarkdown2.8 KB
examples/08-clean-standard.mdmarkdown2.6 KB
fixtures/01-no-dlq/queue.jsonconfig564 B
fixtures/02-dlq-retention-shorter-than-source/dlq.jsonconfig649 B
fixtures/02-dlq-retention-shorter-than-source/queue.jsonconfig632 B
fixtures/03-maxreceivecount-too-low/dlq.jsonconfig659 B
fixtures/03-maxreceivecount-too-low/queue.jsonconfig637 B
fixtures/04-poison-ages-out-before-dlq/dlq.jsonconfig663 B
fixtures/04-poison-ages-out-before-dlq/queue.jsonconfig648 B
fixtures/05-default-visibility-short-retention/dlq.jsonconfig650 B
fixtures/05-default-visibility-short-retention/queue.jsonconfig631 B
fixtures/06-public-queue-policy/dlq.jsonconfig663 B
fixtures/06-public-queue-policy/queue.jsonconfig908 B
fixtures/07-fifo-dedup-off/dlq.jsonconfig748 B
fixtures/07-fifo-dedup-off/queue.jsonconfig804 B
fixtures/08-clean-standard/dlq.jsonconfig672 B
fixtures/08-clean-standard/queue.jsonconfig1.0 KB
tests/README.mdmarkdown4.2 KB
tests/_audit.pyscript21.8 KB
tests/_replay.pyscript720 B
tests/eval/README.mdmarkdown8.4 KB
tests/eval/eval_results.jsonconfig220.4 KB
tests/eval/judge_prompt.mdmarkdown1.4 KB
tests/eval/queues.pyscript7.6 KB
tests/eval/rubric.mdmarkdown2.6 KB
tests/eval/run_eval.pyscript11.5 KB
tests/replay_01_no_dlq.pyscript1.5 KB
tests/replay_02_dlq_retention.pyscript1.5 KB
tests/replay_03_maxreceivecount.pyscript1.3 KB
tests/replay_04_poison_ages_out.pyscript1.6 KB
tests/replay_05_default_visibility.pyscript1.8 KB
tests/replay_06_public_policy.pyscript1.4 KB
tests/replay_07_fifo_dedup_off.pyscript1.4 KB
tests/replay_08_clean_standard.pyscript1.6 KB

FAILURE_MODES.md

View on GitHub
# Failure modes: `sqs-queue-auditor`

This skill is wrong in predictable ways. The list below is the reason it ships with a quality bar that mandates fixture-based replay tests: every failure mode here is a regression vector and gets a test once it shows up in the wild.

## The defining limit: configuration, not behaviour

This skill reads one queue's static attributes. It does not read what the queue is doing. A queue can pass every check in `SKILL.md` and still be dropping messages right now for a reason only the live telemetry shows: a consumer that crashes on a specific payload, a producer that stopped sending, a redrive that is firing constantly. **A clean audit means the configuration is sound, not that the system is healthy.** Every audit says this in its boundary section. Read it as load-bearing, not boilerplate.

## Methodology-level failure modes

### F1. The "is this a processing queue" judgment is supplied, not derived

R1 (no DLQ) only fires when the caller asserts the queue is a processing queue. A pure buffer or fan-out queue with an at-least-once contract enforced elsewhere may legitimately have no dead-letter queue. The audit cannot tell a processing queue from a buffer from the attributes alone; the queue's role is a property of the architecture around it.

**Mitigation in the methodology**: the caller passes `is_processing_queue`. When the role is unknown, the audit should be run as `is_processing_queue=True` (the safe default that surfaces the missing DLQ) and the finding read as "confirm this queue's role" rather than "this is definitely wrong".

**Escalation rule**: if the queue's role is genuinely unknown, surface R1 as a question, not a verdict.

### F2. The R4 age-out estimate is a lower bound, not a guarantee

R4 computes `maxReceiveCount x VisibilityTimeout` as the wall-clock a poison message needs to reach the DLQ, and flags when that exceeds retention. The product is a *lower* bound: it assumes a consumer receives the message roughly once per visibility window. If consumers poll less often, the real time-to-DLQ is longer, which makes the age-out worse, not better. The direction is safe (the flag never under-warns), but the exact margin depends on receive cadence, which is behind the boundary.

**Mitigation in the methodology**: R4 is reported as "poison messages can age out", with the arithmetic shown, so the operator can judge the margin against their actual consumer cadence.

### F3. R3 assumes the DLQ retention is the binding constraint

R3 flags `dlq_retention <= source_retention`. It assumes a message can fail near the end of the source's retention window. If the workload guarantees every message is processed (or fails) within minutes of being sent, a message will never be old enough for the DLQ's shorter retention to bite, and R3 is a false positive in practice. The check is correct for the worst case; the worst case may not occur for a given workload.

**Mitigation in the methodology**: R3 is critical because the worst case is silent total loss of the most diagnostic messages, and the fix (raise DLQ retention to the 14-day max) is cheap and side-effect-free. Prefer the false positive to the silent loss.

### F4. The resource policy is only half the access story

R7 reads the queue's resource `Policy`. The effective set of principals that can `SendMessage` / `ReceiveMessage` is the *union* of that resource policy and every IAM identity policy in the account. A queue with no resource policy at all is not "private": identity policies elsewhere may grant broad access. R7 can only ever flag what the resource policy itself exposes.

**Escalation rule**: a clean R7 is not proof the queue is access-scoped. The IAM-union join is named in every boundary section; closing it needs the account's identity policies, which this skill does not read.

### F5. The low-severity flags cannot be confirmed from the queue

R5 (default visibility), R8 (encryption off), and R9 (FIFO dedup off) each depend on something the queue does not contain: consumer processing time, the data classification, and producer behaviour respectively. They are surfaced as low-severity flags precisely because the skill cannot prove them. Treating a low flag as a confirmed bug is a misread.

## Operational failure modes

### O1. Stale attributes

`GetQueueAttributes` is a point-in-time read. If the queue was reconfigured after the snapshot was taken, the audit describes the old config. `LastModifiedTimestamp` is in the attributes; check it against when the snapshot was pulled.

### O2. The DLQ attributes were not provided

R3 (retention ordering) needs the DLQ's own attributes. When the source queue references a DLQ but the DLQ's `GetQueueAttributes` was not supplied, the retention-ordering check is silently skipped. The audit notes the DLQ is wired but cannot validate its retention.

**Mitigation**: always pull the DLQ's attributes too. Its ARN is in the source queue's `RedrivePolicy.deadLetterTargetArn`.

**Escalation rule**: if R3 could not be evaluated because the DLQ attributes are missing, say so rather than implying the retention ordering is fine.

### O3. Redrive chains and self-references

A DLQ can itself have a `RedrivePolicy` (a chained dead-letter path), and a misconfiguration can point a queue's DLQ at itself. This skill audits one source queue and its immediate DLQ; it does not walk a redrive chain. A multi-hop dead-letter topology needs the cross-resource graph, not a single-queue read.

## When to escalate to a human (summary)

Escalate, or surface as a question rather than a verdict, when **any** of the following is true:

- The queue's role (processing vs buffer) is unknown and R1 fired.
- R3 could not be evaluated because the DLQ attributes were not provided.
- A clean R7 is being read as proof the queue is access-scoped (it is not; the IAM union is unread).
- A low-severity flag (R5, R8, R9) is about to be acted on as a confirmed bug without checking the thing behind the boundary.
- The dead-letter topology is multi-hop (a redrive chain).

Escalation does not mean the agent stops. It means: report the findings, state which checks were deferred and why, name the boundary, and let the human or the next data source close the join.

## How to add a new failure mode here

When a replay test catches a misclassification, or a real-world use surfaces a new pattern, add it under "Methodology-level" or "Operational" with:

1. A short name (`F6`, `O4`, ...).
2. The failure shape, in one sentence.
3. Whatever the methodology already does about it.
4. The escalation rule for it.

Then add a regression test under `tests/` that asserts the audit produces the correct response, even if the response is "defer to the boundary, do not assert a bug".
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall