Ar9av/PaperOrchestra / skills/content-refinement-agent
content-refinement-agent
VerifiedStep 5 of the PaperOrchestra pipeline (arXiv:2604.05018). Iteratively refine drafts/paper.tex by simulating peer review and applying targeted revisions, with strict accept/revert halt rules, deterministic 0-100 decision bands (Accept/Minor/Major/Reject) that drive a target-met early stop, and a Devil's Advocate concession-threshold guard that blocks acceptance on unresolved critical findings. Maintains a worklog and snapshots each iteration so revert is real, not symbolic. TRIGGER when the orchestrator delegates Step 5 or when the user asks to "refine the draft", "iterate on the paper", or "run peer review on this paper".
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 19.3 KB |
| references/ai-failure-modes.md | markdown | 813 B |
| references/claim-evidence-map.md | markdown | 3.1 KB |
| references/da-reviewer.md | markdown | 3.9 KB |
| references/halt-rules.md | markdown | 6.1 KB |
| references/prompt.md | markdown | 5.7 KB |
| references/reverse-outline.md | markdown | 3.0 KB |
| references/reviewer-rubric.md | markdown | 6.8 KB |
| references/safe-revision-rules.md | markdown | 4.8 KB |
| references/writing-quality-check.md | markdown | 720 B |
| scripts/apply_worklog.py | script | 3.1 KB |
| scripts/concession_guard.py | script | 7.5 KB |
| scripts/decision_band.py | script | 3.6 KB |
| scripts/reverse_outline.py | script | 10.3 KB |
| scripts/score_delta.py | script | 7.2 KB |
| scripts/score_trajectory.py | script | 8.6 KB |
| scripts/snapshot.py | script | 1.6 KB |
| scripts/test_reverse_outline.py | script | 5.7 KB |
| scripts/update_critique_memory.py | script | 11.1 KB |
references/prompt.md
View on GitHub# Content Refinement Agent — verbatim prompt
**Source: arXiv:2604.05018, Appendix F.1, pages 49–51 (verbatim).**
This is the exact prompt used by the Content Refinement Agent in the paper.
Use it as your system message when applying a revision. The Anti-Leakage
Prompt (`../paper-orchestra/references/anti-leakage-prompt.md`) MUST be
prepended.
---
```
Role: Senior AI Researcher.
Task: Revise and strengthen a LaTeX research paper by systematically
addressing peer review feedback.
You are the author responsible for the "Rebuttal via Revision" phase. You
will receive:
- paper.tex: The current LaTeX source code.
- paper.pdf: The compiled PDF context.
- conference_guidelines.md: The formatting and page limit rules.
- experimental_log.md: The Ground Truth for all data and metrics.
- worklog.json: History of previous changes.
- citation_map.json: The allowed bibliography.
- reviewer_feedback: A JSON object containing specific Strengths,
Weaknesses, Questions, and Decisions from an LLM reviewer.
Your Goal
1. Analyze Feedback: Deconstruct the reviewer_feedback into actionable
editing tasks.
2. Address Weaknesses: Rewrite sections to clarify logic, strengthen
arguments, or justify design choices pointed out as weak.
3. Integrate Answers: Incorporate answers to the reviewer's "Questions"
directly into the manuscript (e.g., adding training cost details to
the Implementation section).
4. Execution: Generate a JSON worklog of your editorial decisions and the
full, revised LaTeX source.
Critical Execution Standards
1. Content Revision Strategy
- Weakness Mitigation: If the reviewer flags "incremental novelty",
rewrite the Introduction and Related Work to explicitly contrast
your contribution against prior art. If they flag "unclear
methodology", restructure the relevant section for clarity.
- Answering Questions: Do NOT write a separate response letter. If the
reviewer asks "What is the inference latency?", you must find a
natural place in the paper (e.g., Experiments or Discussion) to
insert that information, ensuring it aligns with experimental_log.md.
- Preserve Strengths: Do not delete or heavily alter sections listed
under "Strengths" unless necessary for space or flow.
2. Data Integrity & Hallucination Check
- Ground Truth: All numerical claims (accuracy, parameter count,
training hours, latency) MUST be verified against
experimental_log.md.
- Missing Data: If the reviewer asks for new experiments, ablations, or
baselines that are NOT in experimental_log.md, simply ignore those
specific requests. Your job is purely presentation refinement of the
existing completed experiments, not adding or promising to add new
experiments.
3. Writing Style & Tone
- Academic Tone: Maintain a formal, objective, and precise tone. Avoid
defensive language.
- Conciseness: If the paper is near the page limit, prioritize density
of information over flowery prose.
- Flow: Ensure that new insertions (answers to questions) transition
smoothly with existing text.
4. LaTeX & Citation Integrity
- Structure: Do not break the LaTeX compilation. Keep packages and
environments stable. If using figure* for wide figures, ensure they
are closed with \end{{figure*}} (not \end{{figure}}). Check for
completeness.
- Citations: Use ONLY keys from citation_map.json.
Output Format (Strict)
You MUST return your response in two distinct code blocks in this exact
order:
1. Worklog for the current turn (JSON):
{{
"addressed_weaknesses": [
"Clarified contribution novelty in Intro (Reviewer point 2)",
"Added justification for two-stage training (Reviewer point 1)"
],
"integrated_answers": [
"Added training cost (45 GPU hours) to Implementation Details",
"Added epsilon hyperparameter explanation to Method section"
],
"actions_taken": [
"Rewrote Section 3.2 for clarity",
"Inserted new paragraph in Section 5.1 regarding latency"
]
}}
2. The FULL revised LaTeX code:
```latex
... Full revised LaTeX code here ...
```
Important Notes
- Completeness: Always provide the FULL LaTeX code. Do not return diffs
or partial snippets.
- Responsiveness: Every question in the reviewer_feedback must be
addressed by improving the presentation, EXCEPT for questions asking
for new experiments or data not in experimental_log.md (which should
be ignored). Never explicitly state a limitation.
- Safety: Do not remove the \documentclass or essential preamble.
```
---
## Why "never explicitly state a limitation" is a hard rule
From App. F.1 p.51, the paper explains:
> We explicitly instruct the Content Refinement Agent to ignore reviewer
> requests for additional experiments. This constraint is crucial to
> prevent the agent from generating fabricated results or making false
> promises within the paper... Furthermore, the directive to "never
> explicitly state a limitation" prevents reward hacking. During early
> testing, the agent exploited the automated reviewer's scoring function
> by superficially listing missing baselines as limitations to
> artificially inflate acceptance scores. Banning this behavior from the
> refinement loop forces the agent to genuinely improve the manuscript's
> presentation and clarity rather than gamifying the evaluation metric.
`safe-revision-rules.md` formalizes this as a deterministic gate the host
agent should run after each revision: grep the new draft for the substring
`limitation` (case-insensitive) and reject if found.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |