Skip to content

angushushu/shu-academic-writing-skill / core/audit-manuscript

audit-manuscript

Verified

Conduct an independent, adversarial pre-submission audit of an empirical manuscript against editorial fit, claim–evidence alignment, methods and statistics, transparency and reproducibility, theory, citations, and prose. Use for reviewer simulation, desk-rejection risk assessment, red-team review, submission-readiness gates, or producing a prioritized revision memo. Do not use to rewrite the paper, and never resolve an evidence conflict in the authors' favor.

FileKindSize
SKILL.mdskill md11.3 KB
references/editorial-rubric.mdmarkdown4.4 KB
references/reporting-checklist.mdmarkdown3.6 KB
references/source-quality.mdmarkdown4.0 KB

references/source-quality.md

View on GitHub
# Source quality

How to judge the literature a manuscript rests on. The rest of this module audits what the
authors did; this file audits what they *cite*. After the Harvard Psychology department's
guide for concentrators, which in turn draws on Kosslyn, Maher, and Baumeister & Leary.

A citation problem is easy to miss because citations look like evidence. Often they are not.

## Opinion is not evidence

In an empirical field, evidence means research results — data evaluated against a hypothesis,
produced by observation or controlled procedure. An author's considered judgment is not
evidence, however eminent the author, and neither is the fact that many authors share it.

This matters most where a claim has been repeated so often it feels established. Ask of any
load-bearing citation: **does this source report a result, or state a position?** Both can
belong in a paper, but only one can carry an empirical claim.

Watch for the manuscript's own version of this: a linking proposition asserted with a citation
that, on inspection, is a review article agreeing that the proposition is plausible.

## Quantity and preponderance

Contradictory findings are normal. A conclusion should be supported by the weight of the
evidence presented, not by the subset that agrees with it.

- Converging results across studies, labs, and measures beat a single striking finding.
- A single well-designed study is not overturned by one failed replication, and does not
  survive many.
- When a manuscript cites one supportive study for a contested claim, that is a finding.
  Ask what the rest of the literature says, and whether the authors know.

The Harvard guide's own worked example is instructive: Bem & Honorton's 1994 ESP meta-analysis
reported above-chance results; a later analysis across thirty experiments in seven labs did
not. Absent methodological objections to the second, the preponderance of published evidence
does not support the effect. A paper citing only the first would be citing accurately and
concluding wrongly.

## Read what you cite

If a manuscript relies on more than two or three "as cited in" references, that is a finding.
Taking one author's word for what another reported invites drift: a hedged original becomes a
confident secondary summary becomes a flat assertion in the citing paper.

- Every directly cited work should have been read by someone on the author team.
- Textbooks and handbooks are secondary. They summarize and interpret; they are not the source
  of a specific empirical claim.
- Popular press, magazines, and trade books can supply an illustration or a motivating case.
  They cannot carry an empirical claim.
- Where the project's own records flag a source as unobtained or unverified — see the
  bibliographic-gaps section of `project/theory-map.md` — no claim may depend on it.

## Does conclusion strength match evidence strength

The single question that catches the most citation problems. Check in both directions:

- **Overreach:** the cited work supports a narrower, hedged, or differently scoped claim than
  the manuscript attributes to it.
- **Underreach:** the manuscript hedges a claim its sources actually establish, usually
  because the author has not read them closely enough to know.

Also check for author agenda, soundness of the cited methodology, and appropriateness of the
cited analyses — the same four lanes this module applies to the manuscript itself, applied to
its foundations.

## Two prose consequences

Both are common enough to check mechanically:

- **Do not write that anything is proven.** Theories are provisional; evidence supports,
  is consistent with, or fails to support. This is the calibration ladder in
  `core/polish-prose/references/style-guide.md`, applied to other people's work.
- **Do not editorialize about the literature.** "It would be foolish to ignore," "this study
  completely failed," "it is obvious that" — evaluative adjectives substitute for the
  comparison that should be doing the work. Present the evidence and let the weight show.
rea-analysis-optimizationmorluto/reaOptimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-ci-tests-auditmorluto/reaAudit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
rea-tool-designmorluto/reaAudit, investigate, design, or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.Stars: 472kUpdated Claude CodeCodexVerifiedInstall
ponytail-auditDietrichGebert/ponytailQuality audit of the whole repo: bugs, security, real load, missing tests, speed, and what to delete. Most important first.Stars: 194kUpdated Claude CodeCodexVerifiedInstall