Skip to content

Arnie016/codex-prompt-templates / skills/agent-observability

agent-observability

Verified

Add lightweight local observability to Codex/agent workflows. Use when tracking local runs, traces, evals, cost, token usage, cache efficiency, model comparisons, failure rates, or claudetop-style monitoring. Skip when designing production gateway policy, staged guardrails, shadow traffic, model budgets, or trace-to-regression loops; use `$auto-skill-build-agent-reliability-loop`.

Agent Observability

Use lightweight local observability before adopting a full platform.

Minimal Run Ledger

Create:

.agent-runs/
  runs.jsonl
  evals.md
  costs.md

Each runs.jsonl entry should include:

{"ts":"","task":"","model":"","inputs":"","commands":[],"result":"","tokens_est":"","cost_est":"","status":""}

What To Track

  • task type and outcome
  • commands run
  • tests passed/failed
  • estimated tokens and cost
  • repeated failure causes
  • user corrections
  • guardrail or safety decisions

Eval Loop

  1. Define 3-10 realistic tasks.
  2. Run the same task across workflow variants.
  3. Score correctness, time, tokens, and human intervention.
  4. Keep the cheaper path only if quality is comparable.

Skip Or Escalate

When observability needs gateway policy, staged guardrails, shadow traffic, trace-to-regression feedback, or model budgets, switch to $auto-skill-build-agent-reliability-loop. Keep this skill for local ledgers and lightweight eval/cost tracking.

Use full platforms like Future AGI when you need hosted traces, simulations, gateway routing, or production guardrails. Use local ledgers for personal workflows.

For appending or summarizing .agent-runs/runs.jsonl, use $auto-skill-build-agent-run-ledger-dashboard.

For Codex Supercharge artifact grades and subagent/automation handoff status, use $auto-skill-build-supercharge-feedback-loop; it writes a separate local feedback stream under plugins/codex-supercharge/.agent-runs/.

Validation

  • Each tracked run has a task, status, commands/tests, and outcome.
  • Cost/token notes are marked as estimates unless measured.
  • Repeated failures are summarized into a follow-up or eval case.
brainstormingobra/superpowersYou MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation).Stars: 319kUpdated Claude CodeCodexVerifiedInstall
brand-guidelinesanthropics/skillsApplies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply.Stars: 196kUpdated Claude CodeCodexVerifiedInstall
frontend-designanthropics/skillsGuidance for distinctive, intentional visual design when building new UI or reshaping an existing one. Helps with aesthetic direction, typography, and making choices that don't read as templated defaults.Stars: 196kUpdated Claude CodeCodexVerifiedInstall
mcp-builderanthropics/skillsGuide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. Use when building MCP servers to integrate external APIs or services, whether in Python (FastMCP) or Node/TypeScript (MCP SDK).Stars: 196kUpdated Claude CodeCodexVerifiedInstall