alirezarezvani/claude-skills / engineering/skills/agent-run-forensics
agent-run-forensics
VerifiedUse when asked why an agent did something, what a past session or run changed, whether tests really passed, or to verify a session summary. Answers from the recorded transcript, not memory, with trace_finder.py, trace_timeline.py and claim_checker.py.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 5.2 KB |
| assets/claims_example.txt | markdown | 210 B |
| assets/forensic_answer_template.md | markdown | 754 B |
| references/claim_verification.md | markdown | 3.4 KB |
| references/recorded_vs_inferred.md | markdown | 3.9 KB |
| references/trace_sources.md | markdown | 3.8 KB |
| scripts/claim_checker.py | script | 10.2 KB |
| scripts/trace_core.py | script | 35.0 KB |
| scripts/trace_finder.py | script | 7.1 KB |
| scripts/trace_timeline.py | script | 14.3 KB |
references/trace_sources.md
View on GitHub# Trace sources: where agent runs are recorded
The scripts read files the harnesses already write. Nothing is installed, no
recorder runs alongside the agent, and nothing leaves the machine.
## Where each harness records
| Harness | Location | Format | Confidence |
|---|---|---|---|
| Claude Code (interactive, `-p`) | `~/.claude/projects/<cwd with / as ->/<session-id>.jsonl` | one JSON record per line: `user`, `assistant`, plus bookkeeping types | π’ docs + observed |
| Claude Code subagents | `~/.claude/projects/<project>/<session>/subagents/*.jsonl` | same, records carry `isSidechain: true` | π’ observed |
| Claude Code `--output-format stream-json` | wherever you redirected stdout | same message shapes, `session_id` instead of `sessionId` | π’ observed |
| Codex CLI | `~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl` | `session_meta`, `response_item` (`message`, `function_call`, `function_call_output`) | π‘ parser tested on a synthetic fixture only |
`CLAUDE_CONFIG_DIR` moves the Claude Code tree (settings, session history and
plugins move with it); `CODEX_HOME` moves Codex's. The scripts respect both.
## What a record holds, and what it does not
Recorded, per tool call: the tool name and full arguments (command, file path,
old and new strings), the result text, `is_error`, and for shell calls a line
`Exit code N` when the command failed. Claude Code sets `is_error` on a non-zero
exit, so `is_error: false` on a shell call is read as exit 0. When a shell
command changed a file Claude Code was tracking, the result carries a
`bashEditDiff` with that file's path; the scripts treat that as a recorded write.
Not recorded: the filesystem itself. A command's text says what it was meant to
do. `sed -i ... config.yaml` that exited 0 almost certainly rewrote the file, but
the trace holds the command, not the bytes. That is why shell effects are labelled
`inferred` unless the harness diff confirms them.
Not recorded either: the model's hidden reasoning (thinking blocks may be absent
or redacted), anything done outside the agent (your own terminal, CI, a teammate),
and the current turn when a hook fires: the hooks reference notes the transcript
is written asynchronously and can lag the conversation.
## Retention
Claude Code deletes old transcripts after `cleanupPeriodDays` (a settings key;
check your value and the data-retention page). A question about last month may
have no trace left. The finder says so (exit 2) instead of guessing.
## Shapes the parser handles
- Text blocks starting with harness markers (`<system-reminder>`, `<command-name>`,
interrupted-request notices) are dropped from the prose, not from tool calls.
- A heredoc body (`python3 - <<'EOF' ... EOF`) is data, not shell syntax, so a `>`
inside it is never read as a redirect.
- `cd dir && ...` moves the working directory for the rest of that line.
- Per-record `cwd` is used for each call, because the shell's directory can change
between calls.
## Sources
1. Claude Code hooks reference: common input fields, `transcript_path`, asynchronous transcript writes, `additionalContext` for UserPromptSubmit. https://code.claude.com/docs/en/hooks π’
2. Claude Code settings reference: `cleanupPeriodDays` controls how long transcripts are kept. https://code.claude.com/docs/en/settings-reference π’
3. Claude Code settings: `CLAUDE_CONFIG_DIR` relocates settings, session history and plugins. https://code.claude.com/docs/en/settings π’
4. Claude Code headless mode and `--output-format stream-json`. https://code.claude.com/docs/en/headless π’
5. OpenAI Codex CLI repository (session rollouts under `~/.codex/sessions`). https://github.com/openai/codex π‘
6. JSON Lines format. https://jsonlines.org/ π’
7. OpenTelemetry semantic conventions for generative AI (spans for agent and tool calls; the direction recorders are heading). https://opentelemetry.io/docs/specs/semconv/gen-ai/ π’
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| finishing-a-development-branchobra/superpowersUse when implementation is complete, all tests pass, and you need to decide how to integrate the work | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| systematic-debuggingobra/superpowersUse when encountering any bug, test failure, or unexpected behavior, before proposing fixes | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| test-driven-developmentobra/superpowersUse when implementing any feature or bugfix, before writing implementation code | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| skill-creatoranthropics/skillsCreate new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |