Ar9av/PaperOrchestra / skills/content-refinement-agent
content-refinement-agent
VerifiedStep 5 of the PaperOrchestra pipeline (arXiv:2604.05018). Iteratively refine drafts/paper.tex by simulating peer review and applying targeted revisions, with strict accept/revert halt rules, deterministic 0-100 decision bands (Accept/Minor/Major/Reject) that drive a target-met early stop, and a Devil's Advocate concession-threshold guard that blocks acceptance on unresolved critical findings. Maintains a worklog and snapshots each iteration so revert is real, not symbolic. TRIGGER when the orchestrator delegates Step 5 or when the user asks to "refine the draft", "iterate on the paper", or "run peer review on this paper".
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 19.3 KB |
| references/ai-failure-modes.md | markdown | 813 B |
| references/claim-evidence-map.md | markdown | 3.1 KB |
| references/da-reviewer.md | markdown | 3.9 KB |
| references/halt-rules.md | markdown | 6.1 KB |
| references/prompt.md | markdown | 5.7 KB |
| references/reverse-outline.md | markdown | 3.0 KB |
| references/reviewer-rubric.md | markdown | 6.8 KB |
| references/safe-revision-rules.md | markdown | 4.8 KB |
| references/writing-quality-check.md | markdown | 720 B |
| scripts/apply_worklog.py | script | 3.1 KB |
| scripts/concession_guard.py | script | 7.5 KB |
| scripts/decision_band.py | script | 3.6 KB |
| scripts/reverse_outline.py | script | 10.3 KB |
| scripts/score_delta.py | script | 7.2 KB |
| scripts/score_trajectory.py | script | 8.6 KB |
| scripts/snapshot.py | script | 1.6 KB |
| scripts/test_reverse_outline.py | script | 5.7 KB |
| scripts/update_critique_memory.py | script | 11.1 KB |
scripts/decision_band.py
View on GitHub#!/usr/bin/env python3
"""
decision_band.py — Map a 0-100 overall score to a deterministic decision band.
The reviewer rubric (references/reviewer-rubric.md) produces a weighted 0-100
`overall_score` plus a free-form qualitative `decision`. That free-form label is
advisory; this script computes the *canonical* band the refinement loop reasons
about, so the band is reproducible and never drifts from the number it claims to
summarize.
Default bands (override with flags):
>= 80 Accept
65 .. 79 Minor Revision
50 .. 64 Major Revision
< 50 Reject
The bands give the loop an *absolute* quality target to complement the
*relative* accept/revert delta logic in score_delta.py: a paper can keep
improving relative to its previous self yet still sit in "Major Revision", and
conversely the loop can stop once it reaches "Accept" rather than burning
iterations chasing marginal gains.
Usage:
python decision_band.py --score 74.6
python decision_band.py --score-json workspace/refinement/iter2/score.json
python decision_band.py --score 81 --accept-min 80 --minor-min 65 --major-min 50
Exit codes:
0 band computed (always, on valid input)
2 usage / input error
"""
import argparse
import json
import sys
DEFAULT_ACCEPT_MIN = 80
DEFAULT_MINOR_MIN = 65
DEFAULT_MAJOR_MIN = 50
ACCEPT = "Accept"
MINOR = "Minor Revision"
MAJOR = "Major Revision"
REJECT = "Reject"
# Ordered worst -> best, so callers can compare band strength numerically.
BAND_RANK = {REJECT: 0, MAJOR: 1, MINOR: 2, ACCEPT: 3}
def band_for(score: float,
accept_min: float = DEFAULT_ACCEPT_MIN,
minor_min: float = DEFAULT_MINOR_MIN,
major_min: float = DEFAULT_MAJOR_MIN) -> str:
"""Return the decision band for an overall score. Importable by other scripts."""
if score >= accept_min:
return ACCEPT
if score >= minor_min:
return MINOR
if score >= major_min:
return MAJOR
return REJECT
def main() -> int:
p = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
src = p.add_mutually_exclusive_group(required=True)
src.add_argument("--score", type=float, help="Overall score 0-100")
src.add_argument("--score-json", help="Path to a score.json with an overall_score field")
p.add_argument("--accept-min", type=float, default=DEFAULT_ACCEPT_MIN)
p.add_argument("--minor-min", type=float, default=DEFAULT_MINOR_MIN)
p.add_argument("--major-min", type=float, default=DEFAULT_MAJOR_MIN)
args = p.parse_args()
if not (args.accept_min > args.minor_min > args.major_min):
print("ERROR: thresholds must satisfy accept-min > minor-min > major-min",
file=sys.stderr)
return 2
if args.score is not None:
score = args.score
else:
try:
with open(args.score_json) as f:
data = json.load(f)
score = float(data["overall_score"])
except (OSError, json.JSONDecodeError, KeyError, ValueError, TypeError) as e:
print(f"ERROR: cannot read overall_score from {args.score_json}: {e}",
file=sys.stderr)
return 2
band = band_for(score, args.accept_min, args.minor_min, args.major_min)
out = {
"overall_score": score,
"decision_band": band,
"band_rank": BAND_RANK[band],
"thresholds": {
"accept_min": args.accept_min,
"minor_min": args.minor_min,
"major_min": args.major_min,
},
}
print(json.dumps(out, indent=2))
return 0
if __name__ == "__main__":
sys.exit(main())
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |