zrwangems/dsh-zrwang-plugin / ars/skills/academic-paper-reviewer
academic-paper-reviewer
Flagged模拟同行评审:期刊匹配审稿人 + 3 位动态领域审稿人 + 魔鬼代言人,输出编辑决定(接收/小修/大修/拒稿)与返修路线图,可做方法论专项评审与再审核验。Triggers: review paper, peer review, manuscript review, check methodology, re-review | 审稿, 同行评审, 评审论文, 检查方法, 复审。只做评审,不改写稿件。
references/quality_rubrics.md
View on GitHub> 输出语言(本 DSH 集成强制):所有面向用户的文字、稿件、摘要、评审与报告一律使用**简体中文**;英文术语与引文保持英文。文中的 `zh-TW` / `zh-tw-en` 只是上游语言代码与脚本参数,不代表要用繁体输出。用户明确要求其他语言时以用户为准。
>
> 多角色执行(评审团、十阶段流水线、角色扮演)必须使用 DSH 的 Agent Team 工具(`spawn_teammate`、`team_task_*`、`wait_agent`、`send_message`);当前会话看不到这些工具,说明用户没开 Agent Team,必须先提示用户开启,不得退化成单会话顺序扮演。
# Criterion-Bound Judgement Rubric for Academic Paper Review
## Purpose
This rubric defines what each reviewer must examine and how to make the basis of a judgement inspectable. It does **not** provide a calibrated quality score, a paper-ranking scale, or an acceptance probability.
## Calibration status
Every review must declare one of these states:
- `NOT_CALIBRATED` — the required default. Use this whenever there is no empirical target profile matched to the current domain, article type, venue criteria, rubric version, reviewer/model configuration, and review mode.
- `PROFILE_MEASURED` — reserved for a package-level resolution backed by a replay-valid, hash-bound empirical target profile whose exact target fields and actual completed-panel `execution_topology_sha256` match. Producing or naming a calibration profile does not by itself authorize this status.
Individual reviewer seats always emit `NOT_CALIBRATED`, because the actual
completed-panel topology (including fallbacks) does not exist until all seats
finish. The current Schema 6 package adapter also remains
`NOT_CALIBRATED`; `PROFILE_MEASURED` must not appear in a live review package
until a closed profile schema and replay validator are shipped. This is an
honest implementation boundary, not evidence that calibration is impossible.
Never describe either state as proof that judgements are consistent or reproducible across papers, sessions, fields, venues, or model versions. A directional calibration readout is not an empirical target profile and leaves the status `NOT_CALIBRATED`.
## Required judgement form
For every applicable dimension, report:
| Field | Required content |
|---|---|
| Criterion source | The target venue criterion, reporting standard, article-type expectation, or reviewer configuration item being applied |
| Judgement | `EXCEEDS` / `MEETS` / `PARTLY_MEETS` / `DOES_NOT_MEET` / `NOT_ASSESSED` |
| Evidence anchors | Specific manuscript locations or bounded absence anchors |
| Rationale | How the cited evidence bears on the named criterion |
| Uncertainty or scope limit | Missing information, domain dependence, or reviewer limitation |
| Decision bearing? | Whether this judgement affects the recommendation, with a reason |
These labels are criterion-bound categories, not numbers. Do not convert them into points, weights, a total, a percentage, or a latent ranking. Do not map any total or count of labels mechanically to Accept, Minor Revision, Major Revision, or Reject.
Editorial recommendations instead follow the applicable contract or the qualitative, evidence-anchored rules in `editorial_decision_standards.md`. The recommendation must identify the particular unresolved criteria that make it appropriate.
---
## Dimension 1: Originality
Judge the claimed contribution against the paper's stated field, article type, and target venue. Examine whether the work identifies a defensible gap; distinguishes its theory, method, evidence, or application from relevant prior work; and avoids overstating novelty. Replication and boundary-testing studies can make an original contribution without introducing a new theory.
## Dimension 2: Methodological Rigor
Judge whether the design can answer the stated research question and whether execution and reporting support the inferences made. Apply paradigm- and design-appropriate criteria, including sampling, measurement, validity threats, analysis choices, uncertainty, transparency, and reproducibility where relevant. A standard applies only when its scope matches the paper.
## Dimension 3: Evidence Sufficiency
Judge whether each material claim has evidence of the right type, quality, relevance, and coverage for that claim and field. Consider counter-evidence, triangulation, source provenance, primary versus secondary evidence, and important omissions where relevant.
There is **no universal minimum source count and no universal peer-reviewed-source ratio**. Literature needs vary by field, article type, claim breadth, evidence base, and venue. A review may apply a numeric requirement only when an identified target venue, reporting standard, or protocol explicitly imposes it; cite that authority and do not generalize it beyond its scope.
## Dimension 4: Argument Coherence
Judge whether the problem, gap, research question, method, findings, and implications form a traceable argument. Identify unsupported logical transitions, conclusions that exceed the evidence, unresolved counterarguments, and contradictions. Do not confuse a familiar rhetorical structure with a sound argument.
## Dimension 5: Writing Quality
Judge whether the manuscript communicates its reasoning precisely enough to be reviewed and used. Separate presentation problems from substantive research quality, and do not penalize non-native phrasing when meaning remains clear. Identify only issues that materially affect interpretation, verification, or venue requirements; route copyediting-level points as minor issues.
## Dimension 6: Literature Integration
Judge whether the manuscript identifies and critically integrates the literature needed to establish its question, conceptual lineage, alternatives, and contribution. Coverage is assessed relative to the claims and field, not by a fixed number of references. Missing work should be named or bounded by a clearly described literature area whenever possible.
## Dimension 7: Significance and Impact
Judge whether the claimed theoretical, empirical, practical, or policy implications follow from the evidence and matter for the stated audience. Separate demonstrated significance from speculative future impact; do not require cross-field reach when the target criterion values a focused contribution.
---
## Narrative synthesis
The synthesis must preserve disagreements and non-compensatory weaknesses. A strong judgement on one criterion cannot numerically cancel a failure on another. Report:
1. criteria positively verified;
2. unresolved decision-bearing criteria, with evidence anchors;
3. repairability and the work needed to satisfy each criterion;
4. material uncertainty or reviewer-scope limitations; and
5. the resulting recommendation under the applicable decision standard.
If the evidence does not support a judgement, use `NOT_ASSESSED` or state the uncertainty. Do not manufacture precision by selecting a midpoint or averaging reviewers.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| add-product-e2e-evalpaperclipai/paperclipAdd or extend a Paperclip full-stack runner E2E workflow, fixture, matcher, or report evidence path for local or Daytona execution. | Stars: 115k | Updated | Claude CodeCodex | Verified | Install |
| add-runner-evalpaperclipai/paperclipAdd or extend a Paperclip Runner protocol evaluation definition, roster, assertion, or report fixture with provenance and narrow validation. | Stars: 115k | Updated | Claude CodeCodex | Verified | Install |
| diagnose-why-work-stoppedpaperclipai/paperclipCompatibility entry point for failed, stalled, looping, or over-recovered Paperclip work. Use fix-failed-paperclip-run for the full diagnosis, regression, repair, real verification, and green-PR workflow; honor an explicitly requested diagnosis-only scope. | Stars: 115k | Updated | Claude CodeCodex | Verified | Install |
| fix-failed-paperclip-runpaperclipai/paperclipDiagnose and fix failed, stuck, slow, or incorrectly blocked Paperclip tasks and runs. Use with a task/run URL or failure report to trace the root cause, prove a regression red then green, check second-order effects, verify the real workflow, and deliver a PR with passing checks. | Stars: 115k | Updated | Claude CodeCodex | Verified | Install |