leek/agent-skills / skills/retro
retro
UnverifiedLook back over AI-coding sessions (this one, or recent session logs) and propose improvements to the agent's environment.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 9.9 KB |
| agents/openai.yaml | config | 187 B |
| references/session-selection.md | markdown | 2.5 KB |
| scripts/extract-signal.shexecutable | script | 7.4 KB |
| scripts/select-sessions.shexecutable | script | 15.7 KB |
SKILL.md
View on GitHub---
name: retro
description: Look back over AI-coding sessions (this one, or recent session logs) and propose improvements to the agent's environment.
disable-model-invocation: true
argument-hint: "'this' for the current session, or how many recent session logs to mine (default 50)"
---
# Retro
Turn sessions into a ranked list of concrete improvements to the agent's **environment**, not the code. Read-only analysis; **never modify the log files**. Output a numbered proposal list with one verbatim (redacted) evidence line per finding, then let the user pick which to apply.
**Scope.** `this`, or a run straight after a build or a bug fix: the current session. Its conversation is already in context, so skip steps 1–3 and apply the lenses in step 4 to it. After the context was cleared, find that session's log in step 1 and mine only it. Otherwise: the N most recent sessions, steps 1–6.
## Where the logs live
Claude Code (`~/.claude/projects`, `$CLAUDE_CONFIG_DIR/projects`, and each `~/.claude-profiles/*/projects`), Codex (`~/.codex/sessions`, plain and `.jsonl.zst`), Grok (`~/.grok/sessions`), and agy (`~/.gemini/antigravity-cli`: `history.jsonl` plus one SQLite `.db` per conversation). Two scripts in `<this-skill-dir>/scripts/` read them, read-only: they stream archives without decompressing in place and open SQLite read-only. Store their output in the session scratchpad, never beside the logs.
## 1. Pick the N most recent sessions
Default N = 50 **parent sessions** unless the user gives a number. Run once:
```bash
bash <this-skill-dir>/scripts/select-sessions.sh --out <scratch>/retro --count N --exclude <this session's id>
```
It discovers every harness's logs, groups child logs under their parents (children never take a session slot), excludes sessions with an automation hint (headless runs, panelist briefs, orphan children) into `exclusions.jsonl`, and writes `parents.jsonl` and 7 round-robin `batch_<k>.jsonl` files. Its JSON summary gives counts by harness, children, exclusions by reason, unreadable files, and missing harnesses. Then apply [session selection](references/session-selection.md)'s authorship rules to each selected root's `first_prompt`, and rerun with `--exclude` for any root that is not a human session (`--keep-automated` keeps the hinted ones; `--harness` limits harnesses, for example to spread the sample when the user asks for coverage across them: run once per harness with its share of N and its own `--out` directory, and state the allocation). Selection is complete when the summary's counts match `parents.jsonl` and the exclusions, and you report them.
## 2. Extract signal
Never `cat` a whole log. `bash <this-skill-dir>/scripts/extract-signal.sh <path>` prints one JSONL event per user-role message, queued message, assistant reply (its tail, where closing questions sit), shell command, tool call, and failed tool result, each with its source path and line (SQLite step index for agy). A user-role message is not proof of human authorship: exclude preambles, agent briefs, relayed messages, and quoted earlier corrections from human counts. Surface corrections fast by grepping the `user` and `queued` texts:
`grep -iE "no,|actually|that.?s wrong|don.?t |stop |instead|you should have|i told you|revert|why did you|wrong"`
## 3. Fan out: one subagent per batch
50 sessions won't fit one context. Dispatch one subagent per `batch_<k>.jsonl` **in parallel**, on a cheaper model where the harness lets you pick one (extraction, not judgement), each with an identical brief: its batch file path, the `extract-signal.sh` path, and the lenses in step 4 (in Claude Code with this plugin installed, `subagent_type=leek-skills:session-miner`: log-read-only, skips project CLAUDE.md, and remembers patterns from earlier retros so it can mark them recurring; in other harnesses, dispatch and wait per [harness-tools](../harness-tools/SKILL.md), all at once). Each subagent returns a structured findings list; the orchestrator dedupes across batches and synthesizes.
## 4. What to find
You are proposing improvements to the agent's **environment**, so that future runs go better. Each lens below names what to look for and the moment that triggers it.
**Evidence lenses**: what the transcript shows happening:
1. **Corrections**: the user pushed back ("no", "actually", "that's wrong", "don't do that", reverting/redoing). Highest value, capture every one.
2. **Repeated / errored commands**: a command or tool that errored, or was retried several times before working. Note what finally fixed it.
3. **Repeated setup steps**: the same setup/config/env/login/build/cache-clear rediscovered across more than one session.
4. **Content moments**: something worth an article, tweet, tutorial, diagram, prompt, product idea, or example.
**Environment lenses**: what should change so it stops happening:
5. **Navigation**: how easily did the agent find the right files? Hidden dependencies between files? Would a context pointer help? _Use when_ the session spent a long time hunting for one piece of information.
6. **Automated checks**: could a linter, type check, test, or filesystem check have caught this? Read the repo's own check commands first (`composer.json` / `package.json` scripts, Pint/PHPStan/Larastan config, the CI workflow), so a check that exists but is unwired or silently broken is the finding, not a reinvention. A repo with no **guardrail** (no pre-commit hook and no CI job running its lint, static analysis, or tests) is itself a finding. _Use when_ the agent made a mistake a machine could have caught, or the repo has no guardrail at all.
7. **Coding standards**: should the **reviewer** get a new rule, or an existing one clarified or removed? Classify the violation first. A **mechanical** one (a fixed syntactic pattern, a banned function or facade, an import shape, a file-location rule) gets a deterministic check instead of a prose rule: a custom PHPStan/Larastan or ESLint rule, a Pest arch test, a pre-commit hook, or a CI job, whichever the repo's existing guardrail makes cheapest. Reserve the standards doc for **judgement calls** (cross-file consistency, "matches the surrounding style", anything no check could replace). _Use when_ review failed to catch a mistake.
8. **Steering files**: are there instructions in `CLAUDE.md` / `AGENTS.md` (repo or global) that belong in coding standards or an automated check instead? _Use when_ a steering file is large and unwieldy.
9. **Tool economy**: expensive or chatty tool calls that could be streamlined; custom CLIs or MCP servers that are token-inefficient. _Use when_ one call burned a visible share of the context.
10. **No-ops**: instructions in steering files that don't change the agent's behaviour versus the default. _Use when_ the steering files have grown without pruning.
11. **Information access**: could the agent have been *given* the missing fact? Teed dev-server logs, read-only access to a third-party service, a status endpoint. _Use when_ a crucial piece of information simply wasn't reachable.
## 5. Redaction (mandatory)
In every quoted evidence line, replace emails, API keys, tokens, secrets, passwords, and bearer strings with `[REDACTED]`. Keep quotes short.
## 6. Output: a numbered proposal list
Before proposing a change, read the current destination and compare the observed behavior with its instructions. Recover the loaded skill version/path from the transcript when possible (including plugin cache versions); otherwise record **version unknown**. Compare the session date/version with the relevant fix's history. Classify each candidate as **missing guidance**, **existing guidance ignored**, **already fixed**, or **outside this skill's control**. Repeated evidence from an old version is not a current regression. Already-fixed and out-of-scope items belong in a brief exclusions note; ignored guidance needs an enforcement or clarity improvement, not a duplicate rule.
For each finding, give: the proposal, **the one verbatim (redacted) evidence line it came from** (source path and line, or SQLite step index), the loaded version or **version unknown**, its disposition above, the current destination passage, and a one-sentence why. Label assistant diagnoses and summaries as such rather than human corrections. Destinations:
- a content idea to draft / add to the idea library
- a line to add to `CLAUDE.md` / `AGENTS.md` (name the file)
- a slash command or skill to add or update (name it)
- a hook that should run automatically
- a tool or CLI that should be fixed
- a config or settings change
- **or nothing**, if it was a genuine one-off
**Do not change anything.** Present the list and let the user choose which to apply (a structured question ([harness-tools](../harness-tools/SKILL.md)): one `multiSelect: true` question per destination, up to 4 questions per call and 4 options each, further rounds for the rest; otherwise the user picks by number in chat). Lead with the cross-cutting themes (patterns that recurred across 3+ sessions are the highest-value to act on).
## Reference: where a fix belongs
All work goes through two stages, and they carry very different context pressure. The **implementation** agent bears the most: it explores, writes code, and debugs failures. The **reviewer** bears the least, it receives a diff, so it needs no exploration and usually writes no code.
So **standards belong to the reviewer, not the implementer.** A proposal of the form "tell the agent to always do X while coding" is nearly always better placed as a review rule, where it costs nothing until there is a diff to check.
The destinations, ranked by what they cost:
- `CLAUDE.md` / `AGENTS.md` sit in every agent's context in this repo. Spend lines here incredibly sparingly, and mostly on **context pointers** to other files.
- A coding standards doc is read at review time, not implementation time, cheap, and the right home for most rules.
- Reference docs are free until a pointer reaches them. Look for an existing doc before writing a new one.
- A skill costs its description in context if model-invoked, nothing if user-invoked. Follow `writing-for-agents`.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Unverified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |