notque/vexjoy-agent / skills/programming/programming
programming
FlaggedLanguage rules, workflows, and tooling: Go, Python, Rust, Kotlin, Swift, PHP, TypeScript type checks and debugging, OpenStack Python, MCP local-docs servers, Node.js backend APIs.
references/nodejs-api/webhook-patterns.md
View on GitHub---
description: Webhook processing patterns — signature verification, idempotency, retry handling, and queue integration
---
# Webhook Patterns for Node.js APIs
> **Scope**: Receiving and processing webhooks from Stripe, GitHub, and generic HTTP POST webhooks. Signature verification, idempotency keys, retry-safe handlers, and queue offloading.
> **Version range**: Node.js 18+, Express 4.x/5.x
> **Generated**: 2026-04-08
---
## Overview
Two failure phases: verification (signature, replay prevention) and processing (idempotency, error handling). Correct architecture: separate acknowledgment (fast 200) from processing (durable queue).
---
## Pattern Table
| Pattern | Version | Use When | Avoid When |
|---------|---------|----------|------------|
| Raw body middleware before JSON parse | Express 4+ | Stripe/GitHub signature verification | After `express.json()` — body already consumed |
| Idempotency key deduplication | Always | Retry-prone events (payments, emails) | Fire-and-forget notifications |
| Queue offloading (BullMQ) | BullMQ 3+ | Processing > 2 seconds | Sub-100ms handlers |
| `rawBody` middleware | Express 4+ | HMAC signature verification | When body is already parsed |
---
## Correct Patterns
### Raw Body Preservation for Signature Verification
Stripe and GitHub sign the raw body. Once `express.json()` parses it, original bytes are gone and signature check fails.
```typescript
import express from 'express';
import { createHmac, timingSafeEqual } from 'crypto';
// MUST register raw body middleware BEFORE express.json() for webhook routes
app.use('/webhooks/stripe', express.raw({ type: 'application/json' }));
app.use('/webhooks/github', express.raw({ type: 'application/json' }));
// All other routes get JSON parsing
app.use(express.json());
// Stripe webhook handler
app.post('/webhooks/stripe', (req, res) => {
const sig = req.headers['stripe-signature'] as string;
const rawBody = req.body as Buffer; // Buffer, not object
if (!verifyStripeSignature(rawBody, sig, process.env.STRIPE_WEBHOOK_SECRET!)) {
res.status(400).json({ error: 'Invalid signature' });
return;
}
const event = JSON.parse(rawBody.toString('utf-8'));
// Acknowledge immediately, process asynchronously
res.status(200).json({ received: true });
processWebhookEvent(event).catch(console.error);
});
```
**Why**: After `express.json()`, `req.body` is a parsed object. Re-serializing changes whitespace/ordering, breaking the HMAC.
---
### Idempotency with Redis
Store processed event IDs to prevent duplicate processing.
```typescript
import { createClient } from 'redis';
const redis = createClient({ url: process.env.REDIS_URL });
async function processIdempotent(
eventId: string,
handler: () => Promise<void>
): Promise<{ processed: boolean; duplicate: boolean }> {
const key = `webhook:processed:${eventId}`;
const ttl = 24 * 60 * 60; // 24 hours — longer than retry window
// SET NX: only set if key doesn't exist (atomic)
const acquired = await redis.set(key, '1', { NX: true, EX: ttl });
if (!acquired) {
// Already processed — safe to return 200 without re-processing
return { processed: false, duplicate: true };
}
try {
await handler();
return { processed: true, duplicate: false };
} catch (err) {
// Delete key so retry can attempt processing again
await redis.del(key);
throw err;
}
}
// Usage in webhook handler:
app.post('/webhooks/stripe', async (req, res) => {
const event = parseStripeEvent(req.body, req.headers['stripe-signature'] as string);
res.status(200).json({ received: true }); // ACK first
const { duplicate } = await processIdempotent(event.id, async () => {
await handleStripeEvent(event);
});
if (duplicate) {
console.log(`[info] duplicate event ${event.id} — skipped`);
}
});
```
**Why**: Without idempotency, Stripe retries cause duplicate fulfillment. Redis `SET NX` is atomic: no two workers claim the same event.
---
### Queue Offloading for Slow Handlers
For handlers > 2 seconds: respond 200 immediately, push to BullMQ.
```typescript
import { Queue } from 'bullmq';
const webhookQueue = new Queue('webhook-events', {
connection: { url: process.env.REDIS_URL },
defaultJobOptions: {
attempts: 3,
backoff: { type: 'exponential', delay: 1000 },
removeOnComplete: 100, // Keep last 100 for debugging
removeOnFail: 1000,
},
});
app.post('/webhooks/stripe', async (req, res) => {
// Verify signature synchronously (fast)
const sig = req.headers['stripe-signature'] as string;
if (!verifyStripeSignature(req.body, sig, process.env.STRIPE_WEBHOOK_SECRET!)) {
res.status(400).json({ error: 'Invalid signature' });
return;
}
const event = JSON.parse((req.body as Buffer).toString('utf-8'));
// Push to queue — returns in < 5ms
await webhookQueue.add(event.type, event, {
jobId: event.id, // Deduplication: BullMQ skips duplicate jobIds
});
res.status(200).json({ received: true }); // Within Stripe's 30s timeout
});
```
**Why**: Stripe requires 200 within 30s. DB/email/API calls can exceed this. Queue offloading guarantees fast ack. BullMQ `jobId` deduplicates retries.
---
## Pattern Catalog
### Preserve Raw Body for Signature Verification
**Detection**:
```bash
grep -rn 'express\.json()' --include="*.ts" src/
# Check if webhook routes are registered after app.use(express.json())
grep -rn 'app\.post.*webhook' --include="*.ts" src/ -B20 | grep 'express\.json'
```
**Signal**:
```typescript
app.use(express.json()); // Parses ALL bodies first
app.post('/webhooks/stripe', (req, res) => {
// req.body is now a JS object, not the original bytes
const sig = req.headers['stripe-signature'];
stripe.webhooks.constructEvent(req.body, sig, secret); // Always fails!
});
```
**Why this matters**: `express.json()` consumes the stream. `stripe.webhooks.constructEvent()` requires raw bytes to recompute HMAC — always throws signature mismatch.
**Preferred action**: Register `express.raw({ type: 'application/json' })` on webhook paths before `express.json()` on the general middleware stack.
---
### Acknowledge Webhooks Immediately, Process via Queue
**Detection**:
```bash
# Look for awaited DB calls or email sends inside webhook handlers
grep -rn 'app\.post.*webhook' --include="*.ts" src/ -A30 | grep -E 'await.*db|await.*email|await.*stripe|sendMail'
```
**Signal**:
```typescript
app.post('/webhooks/stripe', async (req, res) => {
const event = JSON.parse(req.body.toString());
if (event.type === 'payment_intent.succeeded') {
// Doing heavy work before responding — may exceed 30s timeout
await db.orders.update({ where: { paymentId: event.data.object.id }, data: { status: 'paid' } });
await emailService.sendReceipt(event.data.object.customer_email);
await fulfillmentService.triggerShipment(event.data.object.metadata.orderId);
}
res.status(200).json({ received: true }); // May never reach this if above fails
});
```
**Why this matters**: If any operation throws or exceeds 30s, Stripe retries. Without idempotency: duplicate updates, receipts, shipments. Timeout means no response at all.
**Preferred action**: Respond 200 immediately, push to BullMQ for processing.
---
### Respond Before Queue Push
**Detection**:
```bash
grep -rn 'await.*queue\.add\|await.*enqueue' --include="*.ts" src/
# Queue push before res.status(200) — blocks acknowledgment
grep -rn 'queue\.add' --include="*.ts" src/ -A5 | grep 'res\.status'
```
**Signal**:
```typescript
app.post('/webhooks', async (req, res) => {
await webhookQueue.add('event', req.body); // Blocks if Redis is slow
res.status(200).json({ received: true }); // Only after queue push
});
```
**Why this matters**: If Redis is slow, queue push blocks and may exceed sender's timeout, causing duplicate entries.
**Preferred action**: Respond first, queue second. If Redis is unavailable, log and still return 200 (accept the event, process manually later via retry mechanism):
```typescript
res.status(200).json({ received: true });
// Fire-and-forget the queue push — errors logged but don't affect HTTP response
webhookQueue.add('event', event).catch((err) => {
console.error(`[error] webhook queue push failed for ${event.id}: ${err.message}`);
// Consider dead-letter storage or alerting here
});
```
---
## Error-Fix Mappings
| Error Message | Root Cause | Fix |
|---------------|------------|-----|
| `No signatures found matching the expected signature` | Body parsed by `express.json()` before raw capture | Register `express.raw()` before `express.json()` on webhook routes |
| `Webhook timestamp too old` | Event timestamp > tolerance (default 300s) | Check server clock sync (`ntpdate`); don't replay old test events |
| `Invalid signature` | Wrong webhook secret in env | Verify `STRIPE_WEBHOOK_SECRET` matches dashboard signing secret |
| `BullMQ: Job already exists` | Duplicate job ID on retry | Expected behavior — BullMQ deduplication working correctly |
| Duplicate fulfillment | No idempotency key check | Implement `SET NX` pattern before processing payment events |
| `ECONNREFUSED` on Redis | Redis not running for queue | Check Redis health; implement fallback logging for critical webhooks |
---
## Detection Commands Reference
```bash
# express.json() before webhook routes
grep -n 'express\.json\|app\.post.*webhook' --include="*.ts" src/ -r
# Missing raw body for webhook routes
grep -rn 'app\.post.*webhook' --include="*.ts" src/ -B10 | grep -v 'raw\|express\.raw'
# String comparison for webhook signatures
grep -rn 'signature.*===\|hmac.*===' --include="*.ts" src/
# Heavy processing before 200 response
grep -rn 'await.*db\.\|await.*email\.\|await.*send' --include="*.ts" src/ | grep -i webhook
# Missing idempotency check
grep -rn 'webhook\|stripe\.event\|github\.event' --include="*.ts" src/ | grep -v 'idempotent\|eventId\|processed'
```
---
## See Also
- `auth-patterns.md` — `timingSafeEqual` for signature verification detail
- BullMQ docs: https://docs.bullmq.io/
- Stripe webhook docs: https://docs.stripe.com/webhooks
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-changelog-updatemorluto/reaPrepare or rewrite REA release changelogs and GitHub release notes from pinned Git history, with verified contributor thanks and Release Please synchronization. | Stars: 469k | Updated | Claude CodeCodex | Verified | Install |
| academy-guideanthropics/skillsStop and check this skill before finishing any reply to a question about how to use Claude or a Claude product — it recommends matching courses, tutorials, and use cases from Claude Academy (academy.claude.com), Anthropic's learning hub. Trigger on: "how do I", "how can I", "getting started with", "what can Claude do", "teach me", "learn to use"; questions about artifacts, projects, skills, plugins, connectors, MCP; requests about rolling Claude out to a team, class, or organization; and any ask for training materials, onboarding content, or learning resources. Use it when the user is learning how to use a feature or product — not when they are mid-task and just want the task done. This skill composes with other skills: after consulting product documentation to answer how a Claude feature works, also check here for a matching course or tutorial — a docs-grounded answer and an Academy recommendation belong together. Only recommend on a strong match; never invent Academy content. | Stars: 197k | Updated | Claude CodeCodex | Verified | Install |
| claude-apianthropics/skillsReference for the Claude API / Anthropic SDK — model ids, pricing, params, streaming, tool use, MCP, agents, caching, token counting, model migration. TRIGGER — read BEFORE opening the target file; don't skip because it "looks like a one-liner" — whenever: the prompt names Claude/Anthropic in any form (Claude, Anthropic, Fable, Opus, Sonnet, Haiku, `anthropic`, `@anthropic-ai`, `claude-*`, `us.anthropic.*`, `[1m]`); the user asks about an LLM (pricing/model choice/limits/caching) — never answer from memory; OR the task is LLM-shaped with provider unstated (agent/MCP/tool-definition/multi-agent/RAG/LLM-judge/computer-use; generate/summarize/extract/classify/rewrite/converse over NL; debugging refusals/cutoffs/streaming/tool-calls/tokens). SKIP only when another provider is being worked on (overrides all triggers): OpenAI/GPT/Gemini/Llama/Mistral/Cohere/Ollama named in the query; OR `grep -rE 'openai|langchain_openai|google.generativeai|genai|mistralai|cohere|ollama'` over the project hits (run this grep FIRST if no provider named — don't Read the file). | Stars: 197k | Updated | Claude CodeCodex | Unverified | Install |
| doc-coauthoringanthropics/skillsGuide users through a structured workflow for co-authoring documentation. Use when user wants to write documentation, proposals, technical specs, decision docs, or similar structured content. This workflow helps users efficiently transfer context, refine content through iteration, and verify the doc works for readers. Trigger when user mentions writing docs, creating proposals, drafting specs, or similar documentation tasks. | Stars: 197k | Updated | Claude CodeCodex | Verified | Install |