Skip to content

nWave-ai/nWave / nWave/skills/nw-rr-critique-dimensions

nw-rr-critique-dimensions

Verified

Critique dimensions and scoring for research document reviews

Critique Dimensions for Research Review

Load when reviewing research documents. Apply each dimension systematically.

Dimension 1: Source Selection Bias

Check: contradictory viewpoints included? | Multiple organizations/authors/perspectives? | Geographic/temporal diversity? | Sources truly independent (not circular)?

Flags: 60%+ from single org/author -> critical | All supporting same conclusion without counterpoint -> critical | Single geographic region -> medium | Clustered publication dates -> medium

Dimension 2: Evidence Quality

Check: every major claim cited | sources reputable (peer-reviewed, official, established) | primary over secondary | technical sources recent (5 years) | confidence matches evidence

Flags: uncited claim -> high | blog/forum for factual claim -> high | all secondary sources -> medium | sources >5 years for tech -> medium | high confidence with 1-2 sources -> high

Dimension 3: Replicability

Check: search strategy documented | source selection criteria explicit | methodology transparent | confidence levels with rationale

Flags: no methodology section -> high | vague methodology ("searched the web") -> medium | no confidence ratings -> medium

Dimension 4: Priority Validation

For research driving architectural/strategic decisions.

Q1: Is this the largest bottleneck? (timing/measurement data?) | Q2: Simpler alternatives considered and rejected with evidence? | Q3: Constraint prioritization correct? (>50% solution for <30% problem = flag) | Q4: Key decision data-justified?

Flags: secondary concern addressed while larger exists -> critical | no measurement data for performance -> high | alternatives not documented -> high | prioritization not explicit -> medium

Output template:

priority_validation:
  q1_largest_bottleneck:
    evidence: "{timing data or 'NOT PROVIDED'}"
    assessment: "YES|NO|UNCLEAR"
  q2_simple_alternatives:
    assessment: "ADEQUATE|INADEQUATE|MISSING"
  q3_constraint_prioritization:
    minority_constraint_dominating: "YES|NO"
    assessment: "CORRECT|INVERTED|NOT_ANALYZED"
  q4_data_justified:
    assessment: "JUSTIFIED|UNJUSTIFIED|NO_DATA"
  verdict: "PASS|FAIL"

Dimension 5: Completeness

Check: knowledge gaps documented (what searched, why insufficient) | conflicting info acknowledged with credibility analysis | all required sections present (summary, findings, sources, gaps, citations) | research metadata included

Flags: missing gaps section when gaps exist -> critical | conflicting sources unacknowledged -> high | missing required sections -> high | no metadata -> medium

Review Output Template

review_id: "research_rev_{timestamp}"
reviewer: "nw-researcher-reviewer (Scholar)"

issues_identified:
  source_bias:
    - issue: "{specific description with numbers}"
      severity: "critical|high|medium"
      recommendation: "{actionable fix}"
  evidence_quality:
    - issue: "{specific claim or location}"
      severity: "critical|high|medium"
      recommendation: "{actionable fix}"
  replicability:
    - issue: "{what is missing}"
      severity: "critical|high|medium"
      recommendation: "{actionable fix}"
  priority_validation:
    - issue: "{mismatch description}"
      severity: "critical|high|medium"
      recommendation: "{actionable fix}"
  completeness:
    - issue: "{missing element}"
      severity: "critical|high|medium"
      recommendation: "{actionable fix}"

quality_scores:
  source_bias: 0.00
  evidence_quality: 0.00
  replicability: 0.00
  completeness: 0.00
  priority_validation: 0.00

approval_status: "approved|rejected_pending_revisions"
blocking_issues:
  - "{critical issue 1}"
iteration: 1
max_iterations: 2
add-product-e2e-evalpaperclipai/paperclipAdd or extend a Paperclip full-stack runner E2E workflow, fixture, matcher, or report evidence path for local or Daytona execution.Stars: 115kUpdated Claude CodeCodexVerifiedInstall
add-runner-evalpaperclipai/paperclipAdd or extend a Paperclip Runner protocol evaluation definition, roster, assertion, or report fixture with provenance and narrow validation.Stars: 115kUpdated Claude CodeCodexVerifiedInstall
diagnose-why-work-stoppedpaperclipai/paperclipCompatibility entry point for failed, stalled, looping, or over-recovered Paperclip work. Use fix-failed-paperclip-run for the full diagnosis, regression, repair, real verification, and green-PR workflow; honor an explicitly requested diagnosis-only scope.Stars: 115kUpdated Claude CodeCodexVerifiedInstall
fix-failed-paperclip-runpaperclipai/paperclipDiagnose and fix failed, stuck, slow, or incorrectly blocked Paperclip tasks and runs. Use with a task/run URL or failure report to trace the root cause, prove a regression red then green, check second-order effects, verify the real workflow, and deliver a PR with passing checks.Stars: 115kUpdated Claude CodeCodexVerifiedInstall