Skip to content

ericrisco/rsc-harness / skills/roast-me

roast-me

Verified

Use when someone wants an honest, comedic audit of their own prompting — reads their past agent transcripts, scores prompt quality and compute efficiency, names their worst habits, builds a model-selection cheat sheet. NOT reviewing code an agent produced (that is `code-review`), NOT prompting technique in general (that is `prompt-engineering`).

FileKindSize
SKILL.mdskill md5.4 KB
evals/README.mdmarkdown3.8 KB
evals/cases.yamlconfig5.7 KB
prompts/analyze.mdmarkdown5.5 KB
prompts/compute.mdmarkdown4.8 KB
prompts/roast.mdmarkdown7.6 KB
tools/adapters/__init__.pyscript36 B
tools/adapters/base.pyscript2.4 KB
tools/adapters/claude.pyscript4.2 KB
tools/adapters/codex.pyscript8.0 KB
tools/adapters/gemini.pyscript5.4 KB
tools/adapters/registry.pyscript2.0 KB
tools/extract_prompts.pyscript20.2 KB

tools/adapters/base.py

View on GitHub
"""Base adapter contract for roast-me transcript discovery.

Every runtime adapter implements two methods:
  discover() -> list[Path]  — find local session files; return [] if none found
  parse(path)  -> list[dict] — yield raw prompt-like records from one file

The adapter contract is intentionally minimal. Higher-level normalisation
(truncation, field enrichment, error/correction detection) happens in
extract_prompts.py, not here.

An adapter MUST:
  - Never raise on missing files or unexpected formats — return [] instead.
  - Tag every record with {"runtime": self.RUNTIME_ID}.
  - Return only records that look like user-initiated prompts (not tool results,
    not system messages).
"""

from __future__ import annotations

import re
from abc import ABC, abstractmethod
from pathlib import Path
from typing import Any

#: Shared correction detector — a user message matching this right after a prompt
#: signals the previous turn went wrong. Single source of truth for every adapter
#: (and re-used by extract_prompts.py) so the heuristic never drifts between runtimes.
CORRECTION_RE = re.compile(
    r"\b(no[,.]?\s|wrong|instead|actually|don'?t|shouldn'?t|stop|not that|"
    r"I said|I meant|I asked|that'?s not|please don'?t|why did you|"
    r"you should have|that was wrong|incorrect|try again|redo|"
    r"that broke|you broke|revert|undo)\b",
    re.IGNORECASE,
)


class BaseAdapter(ABC):
    """Contract every runtime adapter must satisfy."""

    #: Short identifier used to tag records and in CLI --runtime flag.
    RUNTIME_ID: str = "unknown"

    @abstractmethod
    def discover(self) -> list[Path]:
        """Return a list of session file paths found for this runtime.

        Must return [] (not raise) when the runtime is not installed or no
        data directory exists.
        """

    @abstractmethod
    def parse(self, path: Path) -> list[dict[str, Any]]:
        """Parse one session file and return a list of raw prompt records.

        Each record must include at minimum:
          - "runtime": str          — the RUNTIME_ID of this adapter
          - "prompt_text": str      — the user's message text
          - "timestamp": float | None  — unix timestamp of the message (or None)
          - "session_file": str     — stringified path of the source file

        Additional fields are welcomed and used by extract_prompts.py if present
        (e.g. "model", "cost_usd").

        Must return [] (not raise) on parse failures.
        """
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 471kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall