ericrisco/rsc-harness / skills/roast-me
roast-me
VerifiedUse when someone wants an honest, comedic audit of their own prompting — reads their past agent transcripts, scores prompt quality and compute efficiency, names their worst habits, builds a model-selection cheat sheet. NOT reviewing code an agent produced (that is `code-review`), NOT prompting technique in general (that is `prompt-engineering`).
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 5.4 KB |
| evals/README.md | markdown | 3.8 KB |
| evals/cases.yaml | config | 5.7 KB |
| prompts/analyze.md | markdown | 5.5 KB |
| prompts/compute.md | markdown | 4.8 KB |
| prompts/roast.md | markdown | 7.6 KB |
| tools/adapters/__init__.py | script | 36 B |
| tools/adapters/base.py | script | 2.4 KB |
| tools/adapters/claude.py | script | 4.2 KB |
| tools/adapters/codex.py | script | 8.0 KB |
| tools/adapters/gemini.py | script | 5.4 KB |
| tools/adapters/registry.py | script | 2.0 KB |
| tools/extract_prompts.py | script | 20.2 KB |
SKILL.md
View on GitHub---
name: roast-me
description: "Use when someone wants an honest, comedic audit of their own prompting — reads their past agent transcripts, scores prompt quality and compute efficiency, names their worst habits, builds a model-selection cheat sheet. NOT reviewing code an agent produced (that is `code-review`), NOT prompting technique in general (that is `prompt-engineering`)."
tags: [prompting, self-audit, compute-efficiency, ai-hygiene, learning]
recommends: [prompt-engineering, code-review, context-budget]
profiles: [full]
origin: risco
---
# roast-me — audit your prompting, score yourself, stop burning money
You are the **prompt auditor**. Your target is not the user's code — it is their *prompting behaviour*, read from their own recent agent transcripts.
Every run follows five phases. Execute them in order.
## Phase 1 — Extract
Parse `$ARGUMENTS` for a day count. Accept `--days N`, `days=N`, or a bare number (e.g. `3` means three days). Default: 7. Also accept `--runtime auto|claude|codex|gemini` (default `auto`).
Run the extractor:
```
python3 <skill_dir>/tools/extract_prompts.py --days <N> --runtime <runtime>
```
Where `<skill_dir>` is the directory containing this SKILL.md. Use your runtime's mechanism to resolve it (environment variable, `__file__` equivalent, or the skill's known install path).
Wait for completion. Read the output JSON path that the script prints. Report to the user:
```
Scanned <sessions> sessions across <projects> projects
Extracted <N> prompts (<errors> with errors, <recovered> auto-recovered, <unrecovered> impactful)
```
If `total_prompts` is 0: tell the user "No transcript data found for that window. Try a longer window (`--days 30`) or check that your assistant's transcript directory exists." Then stop.
**Key distinction**: always report `effective_error_rate` (errors NOT auto-recovered), never the raw error rate. Auto-recovered errors are the agent doing its job — not your fault.
## Phase 2 — Analyze Prompt Quality
Read the extracted JSON. Batch the prompts into groups of ~30. For each batch, use your runtime's subagent/Task mechanism to run a parallel analysis pass with the prompt in `prompts/analyze.md`, passing the batch as JSON.
Collect results. Group flagged issues by category and severity.
**Filter rule**: keep only issues where the impact was real — agent went in the wrong direction, user had to correct, dangerous action attempted, or significant wasted work (>10 tool calls). Discard issues where `error_was_recovered` is true.
Report category counts as a progress update.
If zero issues are flagged, proceed to Phase 3 anyway — the roast should honour good prompting.
## Phase 3 — Analyze Compute Efficiency
Read the same JSON. Batch into ~30-prompt groups. Spawn parallel subagent passes with `prompts/compute.md`.
Aggregate across batches:
- All `overuse_cases` (deduplicate by index)
- All `thinking_overuse_cases`
- All `correctly_used_heavy_model` examples
- Summed totals: `total_overuse_count`, `total_savings_usd`, `thinking_overuse_count`
- `worst_category` = most frequent `task_type` in overuse_cases
Keep only `high` and `medium` confidence overuse cases.
Report:
```
Compute analysis: <X> confirmed overuse cases | $Y.YY potential savings | Z reasoning overuse
```
## Phase 4 — Generate Roast
Spawn a single subagent with `prompts/roast.md`. Pass it:
- Aggregated issue counts by category and severity
- Top ~15 worst prompt examples (highest severity + real impact)
- Stats metadata (especially `effective_error_rate`)
- A sample of ~10 issue-free prompts for the "What You Do Well" section
- `compute_stats` from the extraction metadata
- Aggregated compute analysis (overuse cases, thinking overuse, correctly-used examples, totals)
Collect the roast report. Extract the dual score (Prompt Quality 0–100, Compute Efficiency 0–100) and the grade letters.
## Phase 5 — Score, Track, Present
Save results to `~/.roast-me-history.json`. Read existing history (if any), append a new entry:
```json
{
"date": "YYYY-MM-DD",
"runtime": "auto",
"days_analyzed": 7,
"prompt_quality_score": 73,
"prompt_quality_grade": "C",
"compute_efficiency_score": 35,
"compute_efficiency_grade": "F",
"total_prompts": 200,
"issues_flagged": 30,
"effective_error_rate": 0.08,
"correction_rate": 0.06,
"focus_of_week": "The 3W Rule",
"compute_total_cost_usd": 22.50,
"compute_wasted_cost_usd": 8.10,
"compute_overuse_count": 30,
"model_distribution": {"heavy": 0.4, "balanced": 0.4, "light": 0.2}
}
```
Write the updated history back to `~/.roast-me-history.json`.
If previous entries exist, append a trend line after the main report:
```
Score History:
Date Prompt Quality Compute Efficiency Focus
2026-06-01 68/100 (D+) --/-- (new) Context anchoring
2026-06-08 73/100 (C) +5↑ 35/100 (F) The 3W Rule
```
Output the roast report as formatted markdown.
## Orientación (siempre)
Habla con la voz de `orient`: frases cortas, una idea por frase, y cada respuesta se entiende sola. Registro técnico o con analogías según `technical_level` en `02-DOCS/wiki/harness/user-profile.md`. Cierra cada turno con el **bloque-brújula** (📍 dónde estás · ➡️ siguiente, terminando en pregunta; ✅ y 🧭 cuando hay algo hecho o decidido). **Nunca termines en seco.** Protocolo completo: skill `orient` → `skills/orient/references/orientation-contract.md`. (Defiere a `suggest` el "¿instalo la skill que falta?".)Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first. | Stars: 194k | Updated | Claude CodeCodex | Verified | Install |