alirezarezvani/claude-skills / agent-launcher/skills/grade-iterate
grade-iterate
VerifiedUse when a launched managed agent needs grading against a rubric and improving ('grade my agent', 'is it good enough'). Builds the outcome with outcome_builder.py, reads verdicts with verdict_reader.py and runs held-back evals from eval_scaffold.py.
| File | Kind | Size |
|---|---|---|
| README.md | markdown | 997 B |
| SKILL.md | skill md | 3.5 KB |
| scripts/eval_scaffold.py | script | 3.8 KB |
| scripts/outcome_builder.py | script | 3.6 KB |
| scripts/verdict_reader.py | script | 4.2 KB |
SKILL.md
View on GitHub---
name: grade-iterate
description: "Use when a launched managed agent needs grading against a rubric and improving ('grade my agent', 'is it good enough'). Builds the outcome with outcome_builder.py, reads verdicts with verdict_reader.py and runs held-back evals from eval_scaffold.py."
license: MIT
metadata:
version: 2.11.2
author: Alireza Rezvani
tags: [cma, outcome, rubric, grader, grade-iterate, loop, max-iterations, eval, held-back]
compatible_tools: [claude-code, codex-cli, cursor, antigravity, opencode, gemini-cli]
---
# Phase 3 — Grade → Iterate (the bounded loop)
This is the plugin's **loop**: CMA's `outcome` primitive self-grades the agent's
work in an isolated context and feeds failing verdicts back for the next attempt.
It is **always bounded** by `max_iterations` (1..20) — never "improve forever".
See [`../../references/loops-and-workflows.md`](../../references/loops-and-workflows.md)
and the outcome section of
[`../../references/cma-primitives.md`](../../references/cma-primitives.md).
## Workflow
1. **Define the outcome.**
```bash
python3 scripts/outcome_builder.py \
--sheet ./my-agent/build-sheet.json --max-iterations 5 \
--out ./my-agent/payloads/outcome.json
```
The **rubric is required**; `max_iterations` is clamped to 1..20. Send the
payload as a `user.define_outcome` event (append to the running session).
2. **Read every verdict first.**
```bash
python3 scripts/verdict_reader.py --result ./my-agent/last-verdict.json
```
Tables the rubric outcome and recommends: **SHIP** (`satisfied`), **SHARPEN**
then re-run (`needs_revision`), **ESCALATE** (`max_iterations_reached` /
`failed`), **RESUME** (`interrupted`). With ≤1 iteration left it flips to
"make the single highest-value fix or escalate now".
3. **Loop invariant.** Each iteration must move ≥1 rubric line fail→pass, or the
run halts at the cap and escalates. Don't burn the budget on cosmetic edits.
4. **Once a version passes, run held-back eval.**
```bash
python3 scripts/eval_scaffold.py \
--sheet ./my-agent/build-sheet.json --out ./my-agent/eval.json --concurrency 5
```
Held-back cases (never seen during iteration) run in parallel, capped at the
25-thread CMA ceiling, each graded against the same rubric.
5. **Decide.** SHIP as v0, or promote to a scheduled deployment (Phase 4). Record
the verdict on the goal: `goal_state.py set --phase run-without-you`.
## Hard rules
- **Bounded, always.** No outcome without a `max_iterations` cap.
- **Read the verdict before acting.** The grader's explanation drives the next move.
- **Held-back cases are held back.** Never grade generalization on cases the agent
already iterated against.
## Forcing-question library (recommend + cite)
1. "What are the 3–5 rubric lines?" *Recommend:* grounded, checkable criteria.
*Cite:* cma-primitives.md (rubric required).
2. "How many iterations before you'd rather look yourself?" *Recommend:* 3–5.
*Cite:* loops-and-workflows.md (bounded loop).
3. "On a fail, sharpen the prompt or the tools?" *Recommend:* whichever rubric line
failed points to. *Cite:* verdict_reader next-move table.
4. "Which cases did the agent NOT see?" *Recommend:* hold back ≥3 for generalization.
*Cite:* this SKILL (held-back eval).
## Tools
- `scripts/outcome_builder.py` — user.define_outcome payload (rubric required, cap 1..20).
- `scripts/verdict_reader.py` — grader result → next move.
- `scripts/eval_scaffold.py` — held-back cases + parallel run plan (≤25 threads).
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| reverse-engineer-anythingmorluto/reaReverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA, and build agent environments (AgentEnv bundles) of web applications from a live instance or a capture, and of Linux desktop apps with a task from a request. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis. | Stars: 472k | Updated | Claude CodeCodex | Verified | Install |
| diagnosing-superpowersobra/superpowersUse when a superpowers session went wrong and your human partner wants to know why — repeated work, ignored plans, stumbles, poor results, a skill that didn't fire, "it took too long", "why is it so expensive", "what is it doing" — or wants to build a bug report for the superpowers maintainers, for the current session or a past one identified by id or path, on any harness. | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| dispatching-parallel-agentsobra/superpowersUse when facing 2+ independent tasks that can be worked on without shared state or sequential dependencies | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| executing-plansobra/superpowersUse when executing an implementation plan in the current session as the implementer yourself — your human partner chose inline execution, or no subagent tool is available | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |