wandb/senpai / .agents/skills/senpai-tool-telemetry
senpai-tool-telemetry
VerifiedAudit recent tool use across local Senpai OpenHands root and recursively delegated conversations without exposing tool arguments. Use for provider comparisons, tool-loop diagnosis, status-poll analysis, error rates, and latency reports.
Senpai tool telemetry
Use this developer-only skill when inspecting a copied or mounted Senpai OpenHands state tree. It is deliberately excluded from the skills installed into advisor and student runtimes.
Run the read-only analyzer from the Senpai repository:
uv run python tools/senpai_tool_telemetry.py \
/path/to/openhands_state \
--hours 12 \
--naive-timezone UTC \
--json /tmp/senpai-tool-telemetry.json
Pass multiple state roots to compare deployments in one report. The command
recursively discovers root and delegated-child events/ directories. It
prints a compact table and writes structured JSON containing per
source/root-or-child-depth/role/model/tool calls, successes, errors, pending
calls, retained tool latency, explicit status-check volume, and identical-call
repetition signals.
OpenHands SDK v1.40 event timestamps may be naive local wall time. Give the
capture host's actual timezone with --naive-timezone, using an IANA name,
UTC, or a fixed offset such as +02:00. For copied roots from different
hosts, add a repeatable override such as
--source-timezone /path/to/cedar=+02:00. A naive timestamp without a matching
default or source override fails clearly; the analyzer never guesses UTC or
the machine running the report. Explicit offsets embedded in an event are
preserved regardless of these options.
Tool arguments are never included in the report. The analyzer creates a short-lived in-memory fingerprint only to identify repeated identical calls. Do not copy raw event bodies into a report merely to recover arguments.
Interpret the output carefully:
get_training_statusandget_job_statuscounts are model-issued tool calls. They do not include cheap controller-internal monitor polls.- Latency is action-to-observation wall time when both timestamps survived.
- Model attribution comes from each conversation's current
base_state.json. If a conversation changed models in place, older calls cannot be attributed more precisely from the event store alone. - A pending call has no matching persisted observation. It may be genuinely active or evidence of an interrupted turn.
- A rapid repeat means the same conversation called the same tool with the same arguments inside the configured repetition window. It is a diagnostic signal, not automatically a bug.
Report evidence before recommendations. Separate provider behavior from runtime failures such as dead terminal sessions, missing observations, or schema errors.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 470k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 470k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 470k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |