estuary/agent-skills / skills/estuary-task-stats
estuary-task-stats
VerifiedCheck task processing statistics to see how much data and how many docs are moving. Use when checking throughput, diagnosing a stalled task, verifying data movement, or reporting on task activity. Use when user says "is data flowing", "how much data", "check throughput", "how many documents", "task not processing", "data not arriving", "stats", "bytes processed", "stalled capture", "no data in destination", "data volume", "task report", "usage report", or "how much data did we process".
flowctl raw stats - Task Processing Statistics
Concepts: Stats show document and byte counts per transaction for a task. They answer "is data actually moving, and how much?" — separate from flowctl catalog status, which only shows control-plane state. A task can show OK status while having zero recent stats (stalled, no new source data, or sync schedule delay).
IMPORTANT: The command is flowctl raw stats, not flowctl stats.
Prerequisites
flowctl must already be authenticated — see the estuary-flowctl-setup skill.
Total Data Over a Period
Materialization — bytes read from source collections ("Data Read" in UI):
flowctl raw stats --task <task> --since 24h | \
jq -s '{
total_gb: ([.[] | select(.materialize != null) | .materialize | to_entries[] | .value.right.bytesTotal // 0] | add // 0 | . / 1073741824 * 10 | round / 10),
total_docs: ([.[] | select(.materialize != null) | .materialize | to_entries[] | .value.right.docsTotal // 0] | add),
transactions: ([.[] | select(.materialize != null)] | length)
}'
Capture — bytes written to Estuary collections ("Data Written" in UI):
flowctl raw stats --task <task> --since 24h | \
jq -s '{
total_gb: ([.[] | select(.capture != null) | .capture | to_entries[] | .value.out.bytesTotal // 0] | add // 0 | . / 1073741824 * 10 | round / 10),
total_docs: ([.[] | select(.capture != null) | .capture | to_entries[] | .value.out.docsTotal // 0] | add),
transactions: ([.[] | select(.capture != null)] | length)
}'
Hourly Breakdown
Useful for spotting gaps, spikes, or batch patterns. Matches the hourly bar chart in the Estuary UI.
Materialization:
flowctl raw stats --task <task> --since 48h | \
jq -s '[.[] | select(.materialize != null) | {
hour: .ts[0:13],
bytes: ([.materialize | to_entries[] | .value.right.bytesTotal // 0] | add),
docs: ([.materialize | to_entries[] | .value.right.docsTotal // 0] | add)
}] | group_by(.hour) | map({
hour: .[0].hour,
gb: (([.[].bytes] | add) / 1073741824 * 10 | round / 10),
docs: ([.[].docs] | add)
})'
Capture:
flowctl raw stats --task <task> --since 48h | \
jq -s '[.[] | select(.capture != null) | {
hour: .ts[0:13],
bytes: ([.capture | to_entries[] | .value.out.bytesTotal // 0] | add),
docs: ([.capture | to_entries[] | .value.out.docsTotal // 0] | add)
}] | group_by(.hour) | map({
hour: .[0].hour,
gb: (([.[].bytes] | add) / 1073741824 * 10 | round / 10),
docs: ([.[].docs] | add)
})'
Hours with no entry = no transactions that hour (expected for gaps, quiet sources, or batch sync schedules).
Stats Structure
Raw entries are nested by task type, then by collection/binding:
{
"ts": "2026-04-10T00:01:00Z",
"capture": { "<collection>": { "out": { "docsTotal": N, "bytesTotal": N } } },
"materialize": { "<collection>": { "right": { "docsTotal": N, "bytesTotal": N },
"left": { "docsTotal": N, "bytesTotal": N },
"out": { "docsTotal": N, "bytesTotal": N } } },
"derivation": { "<transform>": { "input": { "docsTotal": N }, "out": { "docsTotal": N } } }
}
For materializations:
right— docs/bytes read from source Estuary collections ("Data Read" in UI)left— docs/bytes read from the destination (for merging)out— docs/bytes written to the destination
The raw output also contains interval heartbeat entries and ack entries — the select(.materialize != null) / select(.capture != null) filters exclude these.
Diagnosing a Stalled Task
If flowctl catalog status shows OK but data isn't arriving:
# Check for any recent activity
flowctl raw stats --task <task> --since 6h | jq -s '[.[] | select(.materialize != null)] | length'
# → 0 = no transactions in 6h despite OK status
# Run hourly breakdown to find where the gap starts (substitute .capture for capture tasks)
flowctl raw stats --task <task> --since 48h | \
jq -s '[.[] | select(.materialize != null) | {hour: .ts[0:13], gb: ([.materialize | to_entries[] | .value.right.bytesTotal // 0] | add // 0 | . / 1073741824 * 10 | round / 10)}] | group_by(.hour) | map({hour: .[0].hour, gb: ([.[].gb] | add)})'
Common causes of stats silence with OK status:
- Source has no new data (expected for quiet sources)
- Sync schedule delay — materializations batch on a schedule (default: 30 min)
- Connector in retry backoff — check with
flowctl logs --task <task> --since 2h
Related Skills
- estuary-task-health — Single-task health check combining status, stats, logs, and history
- estuary-catalog-status — Check control-plane state (running/failed/disabled)
- estuary-logs — Investigate errors if stats show zero activity
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-changelog-updatemorluto/reaPrepare or rewrite REA release changelogs and GitHub release notes from pinned Git history, with verified contributor thanks and Release Please synchronization. | Stars: 470k | Updated | Claude CodeCodex | Verified | Install |
| academy-guideanthropics/skillsStop and check this skill before finishing any reply to a question about how to use Claude or a Claude product — it recommends matching courses, tutorials, and use cases from Claude Academy (academy.claude.com), Anthropic's learning hub. Trigger on: "how do I", "how can I", "getting started with", "what can Claude do", "teach me", "learn to use"; questions about artifacts, projects, skills, plugins, connectors, MCP; requests about rolling Claude out to a team, class, or organization; and any ask for training materials, onboarding content, or learning resources. Use it when the user is learning how to use a feature or product — not when they are mid-task and just want the task done. This skill composes with other skills: after consulting product documentation to answer how a Claude feature works, also check here for a matching course or tutorial — a docs-grounded answer and an Academy recommendation belong together. Only recommend on a strong match; never invent Academy content. | Stars: 197k | Updated | Claude CodeCodex | Verified | Install |
| claude-apianthropics/skillsReference for the Claude API / Anthropic SDK — model ids, pricing, params, streaming, tool use, MCP, agents, caching, token counting, model migration. TRIGGER — read BEFORE opening the target file; don't skip because it "looks like a one-liner" — whenever: the prompt names Claude/Anthropic in any form (Claude, Anthropic, Fable, Opus, Sonnet, Haiku, `anthropic`, `@anthropic-ai`, `claude-*`, `us.anthropic.*`, `[1m]`); the user asks about an LLM (pricing/model choice/limits/caching) — never answer from memory; OR the task is LLM-shaped with provider unstated (agent/MCP/tool-definition/multi-agent/RAG/LLM-judge/computer-use; generate/summarize/extract/classify/rewrite/converse over NL; debugging refusals/cutoffs/streaming/tool-calls/tokens). SKIP only when another provider is being worked on (overrides all triggers): OpenAI/GPT/Gemini/Llama/Mistral/Cohere/Ollama named in the query; OR `grep -rE 'openai|langchain_openai|google.generativeai|genai|mistralai|cohere|ollama'` over the project hits (run this grep FIRST if no provider named — don't Read the file). | Stars: 197k | Updated | Claude CodeCodex | Unverified | Install |
| doc-coauthoringanthropics/skillsGuide users through a structured workflow for co-authoring documentation. Use when user wants to write documentation, proposals, technical specs, decision docs, or similar structured content. This workflow helps users efficiently transfer context, refine content through iteration, and verify the doc works for readers. Trigger when user mentions writing docs, creating proposals, drafting specs, or similar documentation tasks. | Stars: 197k | Updated | Claude CodeCodex | Verified | Install |