Skip to content

zrwangems/dsh-zrwang-plugin / ars/skills/academic-paper-reviewer

academic-paper-reviewer

Flagged

模拟同行评审:期刊匹配审稿人 + 3 位动态领域审稿人 + 魔鬼代言人,输出编辑决定(接收/小修/大修/拒稿)与返修路线图,可做方法论专项评审与再审核验。Triggers: review paper, peer review, manuscript review, check methodology, re-review | 审稿, 同行评审, 评审论文, 检查方法, 复审。只做评审,不改写稿件。

FileKindSize
SKILL.mdskill md48.9 KB
_shared/ARCHITECTURE.mdmarkdown51.7 KB
_shared/MODE_REGISTRY.mdmarkdown6.7 KB
_shared/agents/compliance_agent.mdmarkdown9.4 KB
_shared/artifact_reproducibility_pattern.mdmarkdown9.8 KB
_shared/benchmark_report.schema.jsonconfig2.7 KB
_shared/benchmark_report_pattern.mdmarkdown8.8 KB
_shared/bibliographic_integrity_signals.mdmarkdown8.1 KB
_shared/collaboration_depth_rubric.mdmarkdown11.5 KB
_shared/compliance_checkpoint_protocol.mdmarkdown8.9 KB
_shared/compliance_report.schema.jsonconfig8.1 KB
_shared/contracts/README.mdmarkdown54.7 KB
_shared/contracts/activity/adjudication_activity_input.schema.jsonconfig18.5 KB
_shared/contracts/activity/adjudication_activity_store.schema.jsonconfig18.7 KB
_shared/contracts/audit/audit_jsonl.schema.jsonconfig5.4 KB
_shared/contracts/audit/audit_sidecar.schema.jsonconfig8.4 KB
_shared/contracts/audit/audit_verdict.schema.jsonconfig5.4 KB
_shared/contracts/audit/cross_document_consistency_advisory.schema.jsonconfig20.4 KB
_shared/contracts/audit/cross_document_consistency_advisory_draft.schema.jsonconfig23.4 KB
_shared/contracts/audit/cross_document_source_manifest.schema.jsonconfig6.3 KB
_shared/contracts/audit/tortured_phrase_advisory.schema.jsonconfig30.9 KB
_shared/contracts/audit/tortured_phrase_snapshot.schema.jsonconfig4.9 KB
_shared/contracts/audit/tortured_phrase_snapshot_manifest.schema.jsonconfig7.3 KB
_shared/contracts/capability/stage_capability_matrix.jsonconfig23.5 KB
_shared/contracts/claim_standing/candidate_ledger.schema.jsonconfig23.1 KB
_shared/contracts/claim_standing/query_plan.schema.jsonconfig12.7 KB
_shared/contracts/claim_standing/query_plan_v1_1.schema.jsonconfig20.3 KB
_shared/contracts/claim_standing/retrieval_input.schema.jsonconfig14.4 KB
_shared/contracts/claim_standing/stance_record.schema.jsonconfig11.7 KB
_shared/contracts/claim_standing/transmission_ledger.schema.jsonconfig7.6 KB
_shared/contracts/cross_model/codex_citation_receipt.schema.jsonconfig4.7 KB
_shared/contracts/cross_model/codex_citation_request.schema.jsonconfig986 B
_shared/contracts/cross_model/promotion_bakeoff_sealed_commitment.schema.jsonconfig1.5 KB
_shared/contracts/cross_model/promotion_bakeoff_sealed_reveal.schema.jsonconfig1.5 KB
_shared/contracts/degradation_registry.jsonconfig42.5 KB
_shared/contracts/evaluator/full.jsonconfig6.3 KB
_shared/contracts/evidence/claim_registry.schema.jsonconfig2.3 KB
_shared/contracts/evidence/claim_registry_coverage_report.schema.jsonconfig4.8 KB
_shared/contracts/evidence/evidence_row.schema.jsonconfig14.3 KB
_shared/contracts/evidence/evidence_row_v1_1.schema.jsonconfig11.4 KB
_shared/contracts/evidence/evidence_row_v1_2.schema.jsonconfig21.8 KB
_shared/contracts/evidence/evidence_row_v1_3.schema.jsonconfig10.7 KB
_shared/contracts/human_subjects/authority_profile_registry.schema.jsonconfig16.7 KB
_shared/contracts/human_subjects/committee_correspondence.schema.jsonconfig9.9 KB
_shared/contracts/human_subjects/content_coverage_advisory.schema.jsonconfig19.9 KB
_shared/contracts/human_subjects/irb_context_record.schema.jsonconfig12.1 KB
_shared/contracts/human_subjects/resolved_authority_context.schema.jsonconfig11.8 KB
_shared/contracts/human_subjects/review_pathway_rule_trace.schema.jsonconfig9.5 KB
_shared/contracts/human_subjects/review_pathway_trace_request.schema.jsonconfig4.8 KB
_shared/contracts/human_subjects/submission_packet_inventory.schema.jsonconfig8.5 KB
_shared/contracts/human_subjects/submission_packet_manifest.schema.jsonconfig32.2 KB
_shared/contracts/passport/audit_artifact_entry.schema.jsonconfig10.8 KB
_shared/contracts/passport/bibliographic_integrity_signal.schema.jsonconfig48.9 KB
_shared/contracts/passport/citation_provenance.schema.jsonconfig4.8 KB
_shared/contracts/passport/citation_verification_summary.schema.jsonconfig8.2 KB
_shared/contracts/passport/claim_audit_result.schema.jsonconfig7.8 KB
_shared/contracts/passport/claim_drift.schema.jsonconfig3.8 KB
_shared/contracts/passport/claim_intent_manifest.schema.jsonconfig6.6 KB
_shared/contracts/passport/constraint_violation.schema.jsonconfig3.6 KB
_shared/contracts/passport/excluded_source_entry.schema.jsonconfig1.7 KB
_shared/contracts/passport/experiment_alignment_result.schema.jsonconfig4.8 KB
_shared/contracts/passport/experiment_provenance_entry.schema.jsonconfig10.0 KB
_shared/contracts/passport/human_read_log.schema.jsonconfig3.8 KB
_shared/contracts/passport/inquiry_ledger_ref.schema.jsonconfig881 B
_shared/contracts/passport/literature_corpus_entry.schema.jsonconfig30.3 KB
_shared/contracts/passport/preregistration_artifact.schema.jsonconfig4.3 KB
_shared/contracts/passport/rejection_log.schema.jsonconfig3.3 KB
_shared/contracts/passport/reset_ledger_entry.schema.jsonconfig7.1 KB
_shared/contracts/passport/run_ledger.schema.jsonconfig8.0 KB
_shared/contracts/passport/standing_constraint_entry.schema.jsonconfig2.6 KB
_shared/contracts/passport/temporal_audit_results.schema.jsonconfig8.8 KB
_shared/contracts/passport/terminal_policies.schema.jsonconfig8.1 KB
_shared/contracts/passport/timeline.schema.jsonconfig4.7 KB
_shared/contracts/passport/uncited_assertion.schema.jsonconfig3.3 KB
_shared/contracts/passport/uncited_audit_failure.schema.jsonconfig4.4 KB
_shared/contracts/passport/user_attested_read_resolution.schema.jsonconfig3.2 KB
_shared/contracts/passport/version_records.schema.jsonconfig4.9 KB
_shared/contracts/patch/block_manifest.schema.jsonconfig2.1 KB
_shared/contracts/patch/legacy/v1_0/revision_patch.schema.jsonconfig5.3 KB
_shared/contracts/patch/revision_patch.schema.jsonconfig8.9 KB
_shared/contracts/pdf/pdf_content_classifier_diagnostic.schema.jsonconfig1.2 KB
_shared/contracts/pdf/pdf_content_classifier_worker.schema.jsonconfig2.2 KB
_shared/contracts/pdf/pdf_read_preflight.schema.jsonconfig6.8 KB
_shared/contracts/re_review/input_manifest.schema.jsonconfig7.6 KB
_shared/contracts/re_review/legacy/v1_0/input_manifest.schema.jsonconfig6.2 KB
_shared/contracts/re_review/legacy/v1_0/precommitment.schema.jsonconfig8.9 KB
_shared/contracts/re_review/legacy/v1_0/traceability.schema.jsonconfig34.2 KB
_shared/contracts/re_review/legacy/v1_0/verdict_record.schema.jsonconfig10.5 KB
_shared/contracts/re_review/precommitment.schema.jsonconfig8.6 KB
_shared/contracts/re_review/traceability.schema.jsonconfig37.7 KB
_shared/contracts/re_review/verdict_record.schema.jsonconfig10.5 KB
_shared/contracts/research_workflow/inquiry_branch_ledger.schema.jsonconfig17.0 KB
_shared/contracts/research_workflow/research_workflow_profile.schema.jsonconfig7.2 KB
_shared/contracts/research_workflow/research_workflow_profile_selection_receipt.schema.jsonconfig3.9 KB
_shared/contracts/review_target/constructive_review_findings.schema.jsonconfig7.0 KB
_shared/contracts/review_target/criteria_registry.schema.jsonconfig4.0 KB
_shared/contracts/review_target/review_criteria_binding_manifest.schema.jsonconfig7.9 KB
_shared/contracts/review_target/review_criteria_source_receipt.schema.jsonconfig5.6 KB
_shared/contracts/review_target/review_target_context.schema.jsonconfig4.5 KB
_shared/contracts/review_target/review_target_declaration.schema.jsonconfig4.6 KB
_shared/contracts/reviewer/full.jsonconfig3.7 KB
_shared/contracts/reviewer/methodology_focus.jsonconfig2.2 KB
_shared/contracts/reviewer/review_panel_provenance.schema.jsonconfig7.8 KB
_shared/contracts/reviewer/review_panel_provenance_carrier.schema.jsonconfig4.1 KB
_shared/contracts/reviewer/review_panel_provenance_input.schema.jsonconfig3.5 KB
_shared/contracts/revision/author_adjudication.schema.jsonconfig7.8 KB
_shared/contracts/revision/author_adjudication_input.schema.jsonconfig1.2 KB
_shared/contracts/revision/claim_strength_drift_disposition.schema.jsonconfig3.6 KB
_shared/contracts/revision/claim_strength_drift_disposition_input.schema.jsonconfig3.1 KB
_shared/contracts/revision/claim_strength_drift_findings.schema.jsonconfig4.2 KB
_shared/contracts/revision/claim_surface_manifest.schema.jsonconfig3.3 KB
_shared/contracts/revision/integrity_correction_authorization.schema.jsonconfig4.3 KB
_shared/contracts/revision/integrity_correction_authorization_input.schema.jsonconfig1.5 KB
_shared/contracts/revision/integrity_correction_list.schema.jsonconfig1.8 KB
_shared/contracts/revision/integrity_pass_receipt.schema.jsonconfig809 B
_shared/contracts/revision/revision_evidence_bundle.schema.jsonconfig4.9 KB
_shared/contracts/revision/revision_roadmap.schema.jsonconfig11.2 KB
_shared/contracts/submission/format_profile.example.yamlconfig971 B
_shared/contracts/submission/format_profile.schema.jsonconfig4.6 KB
_shared/contracts/submission/submission_verification_report.schema.jsonconfig10.3 KB
_shared/contracts/submission/venue_profile.schema.jsonconfig4.1 KB
_shared/contracts/writer/full.jsonconfig3.6 KB
_shared/cross_model_verification.mdmarkdown83.3 KB
_shared/description-original.mdmarkdown1.0 KB
_shared/dsh-runtime.mdmarkdown1.5 KB
_shared/evals_lift_report.schema.jsonconfig5.0 KB
_shared/ground_truth_isolation_pattern.mdmarkdown16.0 KB
_shared/handoff_schemas.mdmarkdown96.9 KB
_shared/human_subjects_authority_registry.jsonconfig49.5 KB
_shared/mode_spectrum.mdmarkdown5.2 KB
_shared/model_tiering.mdmarkdown12.6 KB
_shared/output_language_pair.mdmarkdown7.7 KB
_shared/policy_data/nature_policy.mdmarkdown3.9 KB
_shared/prisma_trAIce_protocol.mdmarkdown10.2 KB
_shared/raise_framework.mdmarkdown7.5 KB
_shared/references/authority_content_coverage_advisory_protocol.mdmarkdown14.0 KB
_shared/references/claim_standing_candidate_ledger_protocol.mdmarkdown19.2 KB
_shared/references/claim_strength_ladder.mdmarkdown5.5 KB
_shared/references/cross_document_consistency_advisory_protocol.mdmarkdown13.5 KB
_shared/references/evidence_row_protocol.mdmarkdown14.4 KB
_shared/references/firm_rules.mdmarkdown11.0 KB
_shared/references/human_subjects_authority_protocol.mdmarkdown13.1 KB
_shared/references/intent_clarification_protocol.mdmarkdown11.9 KB
_shared/references/irb_terminology_glossary.mdmarkdown12.4 KB
_shared/references/per_source_method_weaknesses.mdmarkdown4.2 KB
_shared/references/protected_hedging_phrases.mdmarkdown8.9 KB
_shared/references/psychometric_terminology_glossary.mdmarkdown8.7 KB
_shared/references/review_criteria_consumer_protocol.mdmarkdown10.3 KB
_shared/references/review_form_note.mdmarkdown11.3 KB
_shared/references/review_pathway_rule_trace_protocol.mdmarkdown8.3 KB
_shared/references/routing_core.mdmarkdown5.7 KB
_shared/references/submission_packet_manifest_protocol.mdmarkdown13.6 KB
_shared/references/word_count_conventions.mdmarkdown9.0 KB
_shared/research_workflow_profiles/field_general.jsonconfig1.2 KB
_shared/review_criteria_registry.jsonconfig11.5 KB
_shared/review_criteria_sources/msr-2027-technical-papers.2026-08-24.jsonconfig2.0 KB
_shared/review_criteria_sources/sigsoft-empirical-standards.2026-08-24.jsonconfig1.4 KB
_shared/sprint_contract.schema.jsonconfig19.0 KB
_shared/style_calibration_protocol.mdmarkdown6.9 KB
_shared/templates/codex_audit_multifile_template.mdmarkdown16.7 KB
agents/devils_advocate_reviewer_agent.mdmarkdown42.6 KB
agents/domain_reviewer_agent.mdmarkdown34.0 KB
agents/editorial_synthesizer_agent.mdmarkdown42.4 KB
agents/eic_agent.mdmarkdown28.8 KB
agents/field_analyst_agent.mdmarkdown11.9 KB
agents/methodology_reviewer_agent.mdmarkdown44.6 KB
agents/perspective_reviewer_agent.mdmarkdown34.3 KB
examples/hei_paper_review_example.mdmarkdown33.8 KB
examples/interdisciplinary_review_example.mdmarkdown27.6 KB
examples/subclaim_decomposition_example.mdmarkdown8.3 KB
references/calibration_mode_protocol.mdmarkdown16.1 KB
references/changelog.mdmarkdown2.0 KB
references/editorial_decision_standards.mdmarkdown13.5 KB
references/guided_mode_protocol.mdmarkdown2.6 KB
references/integration_guide.mdmarkdown1.3 KB
references/quality_rubrics.mdmarkdown6.9 KB
references/re_review_mode_protocol.mdmarkdown52.6 KB
references/review_criteria_framework.mdmarkdown6.8 KB
references/review_panel_provenance_protocol.mdmarkdown10.1 KB
references/review_quality_thinking.mdmarkdown3.9 KB
references/reviewer_sprint_prompt_source.mdmarkdown48.5 KB
references/sprint_contract_protocol.mdmarkdown32.7 KB
references/statistical_reporting_standards.mdmarkdown30.4 KB
references/top_journals_by_field.mdmarkdown13.2 KB
scripts/_claim_audit_constants.pyscript13.7 KB
scripts/_markdown_lint_util.pyscript9.0 KB
scripts/_next_verified_at_ms.pyscript6.5 KB
scripts/_skill_lint.pyscript13.6 KB
scripts/_text_similarity.pyscript16.0 KB
scripts/ars_cache_invalidate.pyscript1.3 KB
scripts/ars_mark_read.pyscript18.2 KB
scripts/ars_write_scope_guard.pyscript30.9 KB
scripts/arxiv_client.pyscript9.4 KB
scripts/audit_panel_claims.pyscript17.3 KB
scripts/audit_snapshot.pyscript23.5 KB
scripts/bibliographic_integrity_signals.pyscript32.2 KB
scripts/build_claim_standing_candidate_ledger.pyscript54.3 KB
scripts/build_claim_standing_query_plan.pyscript25.0 KB
scripts/build_content_coverage_advisory.pyscript45.1 KB
scripts/build_cross_document_consistency_advisory.pyscript77.9 KB

_shared/prisma_trAIce_protocol.md

View on GitHub
---
title: PRISMA-trAIce Protocol (v3.4.0 snapshot)
snapshot_date: "2025-12-10"
upstream_source: "https://github.com/cqh4046/PRISMA-trAIce"
upstream_version_commit: "main"
update_policy: "Manual sync. CI freshness check runs weekly; drift emits warning, not merge block."
citation: "Holst D, et al. Transparent Reporting of AI in Systematic Literature Reviews: Development of the PRISMA-trAIce Checklist. JMIR AI. 2025. doi:10.2196/80247"
---

# PRISMA-trAIce Protocol

Verbatim snapshot of the 17-item PRISMA-trAIce checklist from the GitHub canonical source (`cqh4046/PRISMA-trAIce`), dated 2025-12-10, with per-item ARS check procedures and tier-based behaviour.

**Used by**: `compliance_agent` (Task 8), `scripts/check_prisma_trAIce_freshness.py` (Task 12).

> ⚠️ **Upstream sync warning.** If `cqh4046/PRISMA-trAIce` updates, a freshness CI check emits an annotation but does not block merges. Maintainers must manually re-sync this file and bump `snapshot_date` + `upstream_version_commit`. See `scripts/check_prisma_trAIce_freshness.py`.

## Status disclaimer

**PRISMA-trAIce is a foundational proposal, not a Delphi-consensus standard.** Holst et al. (2025) explicitly characterize their checklist as a "well-reasoned albeit preliminary guideline," developed via systematic adaptation of PRISMA 2020 rather than "a formal, broad-based Delphi study or consensus meeting," and note that the 17 items "have not yet been empirically validated across diverse research contexts." The authors frame PRISMA-trAIce as a "foundational proposal" and a "living standard" intended for the research community to "immediately adopt and refine."

ARS adopts the authors' Mandatory / Highly Recommended / Recommended / Optional tier system as defined upstream, and treats Mandatory failures as Stage 2.5 / 4.5 blocks (per the Tier legend below). This block-on-Mandatory choice follows the authors' own argument that the risk of non-transparent AI use outweighs the risk of adopting a preliminary guideline. Future revisions of the upstream living standard are pulled in via the existing freshness CI (`scripts/check_prisma_trAIce_freshness.py`).

> See Holst D, et al. JMIR AI. 2025. doi:[10.2196/80247](https://doi.org/10.2196/80247) — Discussion and Limitations sections.

## Tier legend

| Tier | Behaviour when item FAILs |
|---|---|
| Mandatory (M) | **Block**: pipeline halts; user must backfill, retreat a stage, or use the 3-round override ladder (see [`_shared/compliance_checkpoint_protocol.md#override-ladder-3-round-friction`](./compliance_checkpoint_protocol.md#override-ladder-3-round-friction)) |
| Highly Recommended (HR) | **Warn**: surfaced at checkpoint; user can skip |
| Recommended (R) | **Info**: logged in compliance_report, shown as dashboard bullet |
| Optional (O) | **Info**: same as R |

> **Gap-tag vocabulary.** Tags `[MATERIAL GAP]`, `[WEAK EVIDENCE]`, `[GAP]` used in this protocol's `reason` fields are defined in [`_shared/compliance_checkpoint_protocol.md#canonical-gap-tag-vocabulary`](./compliance_checkpoint_protocol.md#canonical-gap-tag-vocabulary).

## Stage assignment

| Checked at | Items |
|---|---|
| Stage 2.5 (Methods) | M1, M2, M3, M4, M5, M6, M7, M8, M9, M10 |
| Stage 4.5 (remaining) | T1, A1, I1, R1, R2, D1, D2 |

## Items

### T1 — Title
**Tier:** Optional
**Original (upstream verbatim):** "Indicate the use of AI in the title or subtitle if it played a substantial role."
**ARS check procedure:** If `user_metadata.ai_tools_used[].stage` includes any of `{screening, data_extraction, synthesis, writing}` (i.e. substantial involvement), verify manuscript title OR subtitle mentions AI. If none of the AI tools are substantial, pass automatically.
**Pass criterion:** Substring match for one of: "AI-assisted", "AI-augmented", "artificial intelligence", "LLM-assisted", or language-equivalent.

### A1 — Abstract
**Tier:** Optional
**Original:** "Briefly summarize the AI tool(s) used, the review stage(s) where they were applied."
**ARS check procedure:** Locate manuscript abstract. Verify it names ≥1 AI tool AND the review stages.
**Pass criterion:** Both named. Absence of either → fail.

### I1 — Introduction
**Tier:** Recommended
**Original:** "Briefly state the rationale for using AI tools for specific tasks."
**ARS check procedure:** In introduction, locate one sentence or more that explains why AI was chosen for its task.
**Pass criterion:** Rationale present and tied to ≥1 specific AI task.

### M1 — Protocol registration
**Tier:** Mandatory
**Original:** "State if the use of AI was pre-specified in a protocol, and document deviations."
**ARS check procedure:** Check `material_passport.protocol_registration` field OR manuscript §Methods for explicit pre-registration statement. If AI was added mid-review, check for a "deviation from protocol" paragraph.
**Pass criterion:** Either (a) pre-registration clearly stated, or (b) deviation explicitly disclosed.

### M2 — Tool identification and access
**Tier:** Mandatory
**Original:** "Specify name, version, and provider/developer for each tool, with access details."
**ARS check procedure:** For each entry in `user_metadata.ai_tools_used`, require `name`, `version`, `developer`. If tool is open-source, require repository URL or DOI.
**Pass criterion:** All three fields non-empty for every tool.

### M3 — Purpose and stage
**Tier:** Mandatory
**Original:** "Describe the specific SLR stage and the precise task the AI performed."
**ARS check procedure:** For each tool, verify stage-level task description exists (not just "used AI for screening").
**Pass criterion:** Per-tool, per-stage prose description of ≥1 sentence in manuscript §Methods.

### M4 — Input data
**Tier:** Mandatory
**Original:** "Describe data provided to the AI (e.g., training data for fine-tuning)."
**ARS check procedure:** If any tool was fine-tuned or custom-trained, verify dataset description (source, size, preprocessing). If no fine-tuning, verify this is explicitly stated.
**Pass criterion:** Either (a) dataset described, or (b) explicit "no fine-tuning performed" statement.

### M5 — Output data
**Tier:** Mandatory
**Original:** "Describe the output format (e.g., JSON, scores) and any automated post-processing."
**ARS check procedure:** Verify manuscript §Methods documents output format per tool + any post-processing pipelines.
**Pass criterion:** Format + post-processing both described.

### M6 — Prompt engineering
**Tier:** Mandatory
**Original:** "Report prompt engineering process, full prompts, key parameters for LLMs."
**ARS check procedure:** If any tool is an LLM, verify full prompt text is included (main text or supplementary) AND model parameters (temperature, top_p, max tokens, model version) are documented.
**Pass criterion:** All three present for every LLM tool.

### M7 — Operational details
**Tier:** Highly Recommended
**Original:** "Describe key operational settings, algorithms, or configurations for non-LLM AI."
**ARS check procedure:** For classical ML tools (e.g. ASReview, Abstrackr), verify algorithm + hyperparameters documented.
**Pass criterion:** Algorithm name + at least 2 hyperparameters.

### M8 — Human-AI interaction
**Tier:** Mandatory
**Original:** "Describe human interaction (number of reviewers, verification process)."
**ARS check procedure:** Verify manuscript §Methods states reviewer count, qualifications, and oversight mechanism (e.g., independent dual screening, adjudication).
**Pass criterion:** Reviewer count + oversight mechanism both present.

### M9 — AI performance evaluation methods
**Tier:** Mandatory
**Original:** "Describe methods used to evaluate AI performance (metrics, reference standards)."
**ARS check procedure:** Verify manuscript §Methods describes what performance metrics were computed (e.g., sensitivity, agreement) and against what gold standard.
**Pass criterion:** Metric + reference standard both stated.

### M10 — Data management
**Tier:** Recommended
**Original:** "Describe data management, storage, privacy measures, and compliance."
**ARS check procedure:** Verify manuscript §Methods addresses data handling (where stored, privacy controls, GDPR/IRB compliance if applicable).
**Pass criterion:** ≥2 of {storage, privacy, compliance} addressed.

### R1 — Study selection (AI-assisted)
**Tier:** Mandatory
**Original:** "In the PRISMA flow diagram/text, distinguish between records handled by AI versus human reviewers."
**ARS check procedure:** Verify PRISMA flow diagram or accompanying text explicitly separates AI-handled vs human-handled records at each screening stage.
**Pass criterion:** Explicit AI/human split in flow diagram numbers OR prose.

### R2 — AI performance metrics (results)
**Tier:** Mandatory
**Original:** "Report results of any AI performance evaluations (e.g., agreement rates)."
**ARS check procedure:** Verify manuscript §Results reports numeric performance outcomes referenced by M9.
**Pass criterion:** ≥1 quantitative metric (% agreement, Cohen's κ, sensitivity, etc.).

### D1 — Limitations of AI use
**Tier:** Recommended
**Original:** "Discuss limitations (technical issues, biases, hallucinations) and their impact."
**ARS check procedure:** Verify manuscript §Discussion has ≥1 paragraph specifically on AI limitations (not generic review limitations).
**Pass criterion:** Dedicated AI-limitations paragraph of ≥3 sentences.

### D2 — Implications of AI use
**Tier:** Optional
**Original:** "Briefly discuss the experience of using AI tools for future reviews."
**ARS check procedure:** Verify manuscript §Discussion reflects on the usability/experience of AI for this review, forward-looking.
**Pass criterion:** ≥1 forward-looking sentence about AI utility.

## Summary counts

- Mandatory: 10 — M1, M2, M3, M4, M5, M6, M8, M9, R1, R2
- Highly Recommended: 1 — M7
- Recommended: 3 — I1, M10, D1
- Optional: 3 — T1, A1, D2

Total: 17 items.

> **ARS promotion note.** Upstream's README header summary says "Mandatory: 9". Per-item tier annotations in upstream (and mirrored here) mark **both R1 and R2 as Mandatory**, summing to 10. ARS treats the per-item annotation as authoritative, since the Results-stage items (R1: AI/human split in flow diagram; R2: AI performance metric reporting) are both operationally block-level for a compliance audit. If upstream clarifies the discrepancy, re-sync.
add-product-e2e-evalpaperclipai/paperclipAdd or extend a Paperclip full-stack runner E2E workflow, fixture, matcher, or report evidence path for local or Daytona execution.Stars: 115kUpdated Claude CodeCodexVerifiedInstall
add-runner-evalpaperclipai/paperclipAdd or extend a Paperclip Runner protocol evaluation definition, roster, assertion, or report fixture with provenance and narrow validation.Stars: 115kUpdated Claude CodeCodexVerifiedInstall
diagnose-why-work-stoppedpaperclipai/paperclipCompatibility entry point for failed, stalled, looping, or over-recovered Paperclip work. Use fix-failed-paperclip-run for the full diagnosis, regression, repair, real verification, and green-PR workflow; honor an explicitly requested diagnosis-only scope.Stars: 115kUpdated Claude CodeCodexVerifiedInstall
fix-failed-paperclip-runpaperclipai/paperclipDiagnose and fix failed, stuck, slow, or incorrectly blocked Paperclip tasks and runs. Use with a task/run URL or failure report to trace the root cause, prove a regression red then green, check second-order effects, verify the real workflow, and deliver a PR with passing checks.Stars: 115kUpdated Claude CodeCodexVerifiedInstall