ashaychangwani/imprint / .claude/skills/imprint-reteach-audit
imprint-reteach-audit
VerifiedRe-teach one or more imprint sites from their existing recordings and verify them with `imprint audit`, including the concurrency rules, background-monitoring recipe, and the known failure modes.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 3.8 KB |
SKILL.md
View on GitHub---
name: imprint-reteach-audit
version: 1.0.0
description: Re-teach one or more imprint sites from their existing recordings and verify them with `imprint audit`, including the concurrency rules, background-monitoring recipe, and the known failure modes.
triggers:
- reteach site
- re-teach
- teach from scratch
- audit site
- rebuild tools
- verify site
allowed-tools:
- Bash
- Read
---
# Re-teach + audit imprint sites
Use this to rebuild a site's tools from its existing recording and confirm they work.
## Commands
```bash
# Re-teach (wipes compiled artifacts, PRESERVES sessions/, recompiles from the
# newest raw recording, tracing on, compile concurrency 2):
IMPRINT_TEACH_TIMEOUT=30m bash scripts/teach-from-scratch.sh <site>
# Audit (spawns a headless claude that calls every compiled tool live and grades
# correct/broken; exit 0 = PASS, default threshold 95%):
bun run src/cli.ts audit <site>
```
## Hard rules
- **Compile concurrency is 2 — never raise it, and never run two teaches at once.** Each teach already runs 2 compile agents; two parallel teaches = 4 concurrent Opus agents → trips rate limits. **Re-teach sites strictly sequentially.** Avoid overlapping a teach with an audit for the same reason.
- **Heavy search tools need `IMPRINT_TEACH_TIMEOUT=30m`+** (batchexecute / multi-filter search; the default is too tight once param-fidelity verification runs).
- Teach is long: **~40–75 min per site**, dominated by the heavy tools' `compile.generate`. Run it in the background and monitor.
- To re-teach several sites + audit each as one hands-off job, loop them sequentially in a single background driver script (teach → audit → next).
## Monitoring a background teach (the log is spinner-heavy)
```bash
# strip ANSI + collapse \r redraws + drop spinner frames
sed -E 's/\x1b\[[0-9;?]*[a-zA-Z]//g' LOG | tr '\r' '\n' \
| grep -aviE '◐|◓|◑|◒|◇|◆|●|Triaging|Replaying|Compiling •|cycle [0-9]/5|^\s*$|^│|^├'
```
Watch for: shared modules `built + verified` vs `pruning`; per-tool `index.ts` appearing under `~/.imprint/<site>/<tool>/`; final `Done! N tools ready`. Avoid grepping `cycle`/`Compiling •` lines raw — they are giant `\r`-concatenated blobs.
## Known failure modes
- **Valid shared module pruned for "must export a transform function"** — was a bun stale-`.ts`-import-cache bug; fixed in `src/imprint/prereq-builder.ts` (`importModuleFresh` copies the module to a unique sibling path before re-importing, defeating the cache). If it recurs, the fix is there.
- **Audit grader non-determinism on a no-op param.** A param that legitimately has no observable effect (e.g. a `brand` filter where two brands share the same data) can be graded `correct` one run and `tool_broken` the next, flipping a site between PASS and ~85%. The tools still return correct data — **re-audit once to confirm it's variance before treating it as a defect** (and report honest numbers, don't re-roll just to pass).
- **A network outage during the replay/capture stage corrupts the session** — events time out with 0 requests captured (the captured-count plateaus). Replay normally takes ~3 min; if it's dragging for many minutes with a flat capture count, the network dropped. Re-teach when the connection is stable. Diagnose via the per-event capture trajectory in the teach log (`Replaying event N/M (K requests captured)`).
## Verify + expose for live testing
- PASS = audit exit 0 and score ≥ 95% (`graded N of N invocations`).
- The MCP registration `imprint mcp-server <site>` is a **command pointer** — it re-reads `~/.imprint/<site>/` on every spawn, so it always serves the latest teach. After a re-teach, a *running* Claude session must reconnect (`/mcp`) or start fresh to pick up new tools; a new session needs nothing.
- Confirm what's served: `claude mcp list` (look for `imprint-<site>` ✓ Connected) and `claude mcp get imprint-<site>`.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 196k | Updated | Claude CodeCodex | Verified | Install |