Arnie016/codex-prompt-templates / skills/agent-observability
agent-observability
VerifiedAdd lightweight local observability to Codex/agent workflows. Use when tracking local runs, traces, evals, cost, token usage, cache efficiency, model comparisons, failure rates, or claudetop-style monitoring. Skip when designing production gateway policy, staged guardrails, shadow traffic, model budgets, or trace-to-regression loops; use `$auto-skill-build-agent-reliability-loop`.
Agent Observability
Use lightweight local observability before adopting a full platform.
Minimal Run Ledger
Create:
.agent-runs/
runs.jsonl
evals.md
costs.md
Each runs.jsonl entry should include:
{"ts":"","task":"","model":"","inputs":"","commands":[],"result":"","tokens_est":"","cost_est":"","status":""}
What To Track
- task type and outcome
- commands run
- tests passed/failed
- estimated tokens and cost
- repeated failure causes
- user corrections
- guardrail or safety decisions
Eval Loop
- Define 3-10 realistic tasks.
- Run the same task across workflow variants.
- Score correctness, time, tokens, and human intervention.
- Keep the cheaper path only if quality is comparable.
Skip Or Escalate
When observability needs gateway policy, staged guardrails, shadow traffic,
trace-to-regression feedback, or model budgets, switch to
$auto-skill-build-agent-reliability-loop. Keep this skill for local ledgers
and lightweight eval/cost tracking.
Use full platforms like Future AGI when you need hosted traces, simulations, gateway routing, or production guardrails. Use local ledgers for personal workflows.
For appending or summarizing .agent-runs/runs.jsonl, use
$auto-skill-build-agent-run-ledger-dashboard.
For Codex Supercharge artifact grades and subagent/automation handoff status,
use $auto-skill-build-supercharge-feedback-loop; it writes a separate local
feedback stream under plugins/codex-supercharge/.agent-runs/.
Validation
- Each tracked run has a task, status, commands/tests, and outcome.
- Cost/token notes are marked as estimates unless measured.
- Repeated failures are summarized into a follow-up or eval case.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| brainstormingobra/superpowersYou MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation). | Stars: 319k | Updated | Claude CodeCodex | Verified | Install |
| brand-guidelinesanthropics/skillsApplies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |
| frontend-designanthropics/skillsGuidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |
| mcp-builderanthropics/skillsGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK). | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |