Skip to content

HezaoHezao/poirot / poirot/backend/agents/skill/builtin_skills/research/research-paper-writing

research-paper-writing

Verified

ML paper pipeline: experiment design to submission.

Research Paper Writing Pipeline

Overview

End-to-end pipeline for producing publication-ready ML/AI research papers targeting NeurIPS, ICML, ICLR, ACL, AAAI, COLM. Covers the full research lifecycle: experiment design, execution, analysis, paper writing, review, revision, and submission.

This is not a linear pipeline — it is an iterative loop. Results trigger new experiments. Reviews trigger new analysis.

When to Use

  • User is writing an ML/AI research paper for a top venue
  • User needs help with experiment design, execution, or analysis
  • User wants feedback on a draft
  • User is preparing a submission package

Pipeline Phases

Phase 0: Project Setup → Phase 1: Literature Review
       │                        │
       ▼                        ▼
Phase 2: Experiment      Phase 5: Paper Drafting ◄──┐
       Design                  │                    │
       │                       ▼                    │
       ▼                 Phase 6: Self-Review       │
Phase 3: Execution            & Revision ───────────┘
       & Monitoring              │
       │                         ▼
       ▼                   Phase 7: Submission
Phase 4: Analysis

Phase 0: Project Setup

  • Define research question and hypothesis
  • Identify target venue + deadline
  • Set up project structure:
    project/
    ├── experiments/
    ├── data/
    ├── src/
    ├── paper/
    │   ├── main.tex
    │   ├── figures/
    │   └── references.bib
    └── README.md
    
  • Initialize git repo, set up environment

Phase 1: Literature Review

  • Use arxiv skill to find related work
  • Use web_search for non-arXiv papers (Semantic Scholar, Google Scholar)
  • Use browse_page to read key papers in full
  • Build a references.bib with all cited works
  • Identify the gap your work fills

Phase 2: Experiment Design

  • Define baselines and comparison methods
  • Choose datasets and evaluation metrics
  • Design ablation studies
  • Plan computational budget
  • Write pre-registration document (optional but recommended)

Phase 3: Execution & Monitoring

# Run experiments
python src/train.py --config configs/exp1.yaml

# Monitor with logging
python src/train.py --config configs/exp1.yaml --log-dir runs/exp1

# Track experiments
python src/eval.py --checkpoint runs/exp1/best.pt --eval-set test
  • Log all hyperparameters, seeds, and environment details
  • Save checkpoints for reproducibility
  • Run each experiment with multiple seeds (3-5)

Phase 4: Analysis

  • Aggregate results across seeds
  • Compute statistical significance (paired t-test, bootstrap CI)
  • Generate comparison tables and plots:
    python src/plot.py --results runs/ --output paper/figures/
    
  • Run ablation analysis
  • Identify surprising findings (investigate, don't hide)

Phase 5: Paper Drafting

Follow venue template structure:

  1. Abstract — problem, method, key result, impact (write last)
  2. Introduction — motivation, contribution summary, roadmap
  3. Related Work — position within literature (from Phase 1)
  4. Method — approach, architecture, training procedure
  5. Experiments — setup, main results, ablations, analysis
  6. Conclusion — summary, limitations, future work

Writing principles:

  • One idea per paragraph
  • Figures tell the story — design figures first, write text around them
  • Tables for comparisons — main results table + ablation table
  • Reproducibility — include all hyperparameters, release code

Phase 6: Self-Review & Revision

Use academic-paper-review skill to self-review:

  • Read the paper cold (fresh eyes)
  • Check methodology soundness, novelty, reproducibility
  • Identify weaknesses and fix them
  • Get feedback from collaborators

Phase 7: Submission

  • Check venue formatting requirements
  • Verify page limits
  • Anonymize for blind review (if applicable)
  • Prepare supplementary material (code, data, extended results)
  • Submit before deadline (not at 23:59)

Statistical Analysis

# Multiple seeds — compute mean ± std
python3 -c "
import numpy as np
results = [0.85, 0.83, 0.86, 0.84, 0.82]  # per-seed results
print(f'Mean: {np.mean(results):.4f} ± {np.std(results):.4f}')
"

# Paired t-test vs baseline
python3 -c "
from scipy import stats
baseline = [0.80, 0.79, 0.81, 0.78, 0.80]
ours = [0.85, 0.83, 0.86, 0.84, 0.82]
t, p = stats.ttest_rel(ours, baseline)
print(f't={t:.3f}, p={p:.4f}')
"

Pitfalls

  • Single seed: results from one seed are not reliable. Use 3-5 minimum.
  • Cherry-picking: report all results, not just the best seed.
  • No ablations: reviewers will ask "does each component matter?" — answer proactively.
  • Missing related work: reviewers know the field. Cite comprehensively.
  • Unclear contributions: list contributions explicitly in the introduction.
  • Overclaiming: "state-of-the-art" needs evidence across datasets, not one.
  • Last-minute submission: servers crash at deadlines. Submit early.

Dependencies

This skill benefits from: numpy, scipy, matplotlib (analysis + plots). Install via pip install numpy scipy matplotlib.

writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deploymentStars: 322kUpdated Claude CodeCodexVerifiedInstall
xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved.Stars: 198kUpdated Claude CodeCodexVerifiedInstall
shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy.Stars: 119kUpdated Claude CodeCodexVerifiedInstall
childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release.Stars: 118kUpdated Claude CodeCodexVerifiedInstall