MiaoDX/roboclaws / skills/report-performance-analysis
report-performance-analysis
VerifiedUse this skill whenever the user asks whether a Roboclaws live-agent report run is faster, wants to compare Agent SDK/Codex/Claude cleanup reports, asks about model-call work, residual latency, speedup validity, or report performance under provider/network variance. The skill extracts sanitized JSON metrics first and answers only from generated metric/comparison artifacts, not from report.html scraping or raw prompts.
Report Performance Analysis
Use this skill to analyze Roboclaws live-agent report performance honestly under unstable provider and network conditions.
Workflow
- Read
references/metric-contract.md. - Identify the run directory or comparison manifest.
- Run fixed scripts before answering:
- One run:
../../scripts/reports/extract_live_report_metrics.py <run_dir> - Two runs or manifest:
../../scripts/reports/compare_live_report_metrics.py ... - Calibration data:
../../scripts/reports/calibrate_model_latency.py ...
- One run:
- Answer from generated JSON packets. Do not scrape
report.html, raw prompt text, model text, full tool payloads, private evaluator truth, or compact continuation state.
Interpretation Rules
- Lead with quality and effect before wall time.
- Treat single-run output as diagnostic unless an explicit baseline or manifest is supplied.
- Missing token, duration, image, or calibration fields are unavailable, not zero.
- A faster candidate is not a speed win when quality regressed unless the comparison packet records an explicit waiver.
- Call out apples-to-oranges baselines, missing telemetry, uncalibrated normalized model time, and high model/SDK residual latency.
Common Commands
Extract one run:
.venv/bin/python scripts/reports/extract_live_report_metrics.py \
output/path/to/seed-7 \
--output output/report-performance-analysis/metrics.json \
--write-model-call-metrics
Compare two runs:
.venv/bin/python scripts/reports/compare_live_report_metrics.py \
--baseline-run-dir output/baseline/seed-7 \
--candidate-run-dir output/candidate/seed-7 \
--output output/report-performance-analysis/comparison.json
Compare with explicit normalized timing:
.venv/bin/python scripts/reports/calibrate_model_latency.py \
--output output/report-performance-analysis/calibration.json \
output/baseline/seed-7/model_call_metrics.jsonl \
output/candidate/seed-7/model_call_metrics.jsonl
.venv/bin/python scripts/reports/compare_live_report_metrics.py \
--baseline-run-dir output/baseline/seed-7 \
--candidate-run-dir output/candidate/seed-7 \
--calibration output/report-performance-analysis/calibration.json \
--output output/report-performance-analysis/normalized-comparison.json
Compare a manifest:
.venv/bin/python scripts/reports/compare_live_report_metrics.py \
--manifest skills/report-performance-analysis/templates/comparison-manifest.json \
--output output/report-performance-analysis/comparison.json
Answer Shape
Use a compact answer:
Verdict: <accepted speed win | diagnostic only | rejected | blocked>
Quality: <same/better/regressed and why>
Work: <model calls, tool calls, uncached/output/reasoning tokens>
Timing: <wall time, model API time, estimated model work availability, residual>
Limitations: <missing telemetry/calibration/apples-to-oranges/live gate>
Evidence: <metric JSON path or comparison JSON path>
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 469k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 321k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 321k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 321k | Updated | Claude CodeCodex | Verified | Install |