HezaoHezao/poirot / poirot/backend/agents/skill/builtin_skills/research/research-paper-writing
research-paper-writing
VerifiedML paper pipeline: experiment design to submission.
Research Paper Writing Pipeline
Overview
End-to-end pipeline for producing publication-ready ML/AI research papers targeting NeurIPS, ICML, ICLR, ACL, AAAI, COLM. Covers the full research lifecycle: experiment design, execution, analysis, paper writing, review, revision, and submission.
This is not a linear pipeline — it is an iterative loop. Results trigger new experiments. Reviews trigger new analysis.
When to Use
- User is writing an ML/AI research paper for a top venue
- User needs help with experiment design, execution, or analysis
- User wants feedback on a draft
- User is preparing a submission package
Pipeline Phases
Phase 0: Project Setup → Phase 1: Literature Review
│ │
▼ ▼
Phase 2: Experiment Phase 5: Paper Drafting ◄──┐
Design │ │
│ ▼ │
▼ Phase 6: Self-Review │
Phase 3: Execution & Revision ───────────┘
& Monitoring │
│ ▼
▼ Phase 7: Submission
Phase 4: Analysis
Phase 0: Project Setup
- Define research question and hypothesis
- Identify target venue + deadline
- Set up project structure:
project/ ├── experiments/ ├── data/ ├── src/ ├── paper/ │ ├── main.tex │ ├── figures/ │ └── references.bib └── README.md - Initialize git repo, set up environment
Phase 1: Literature Review
- Use
arxivskill to find related work - Use
web_searchfor non-arXiv papers (Semantic Scholar, Google Scholar) - Use
browse_pageto read key papers in full - Build a
references.bibwith all cited works - Identify the gap your work fills
Phase 2: Experiment Design
- Define baselines and comparison methods
- Choose datasets and evaluation metrics
- Design ablation studies
- Plan computational budget
- Write pre-registration document (optional but recommended)
Phase 3: Execution & Monitoring
# Run experiments
python src/train.py --config configs/exp1.yaml
# Monitor with logging
python src/train.py --config configs/exp1.yaml --log-dir runs/exp1
# Track experiments
python src/eval.py --checkpoint runs/exp1/best.pt --eval-set test
- Log all hyperparameters, seeds, and environment details
- Save checkpoints for reproducibility
- Run each experiment with multiple seeds (3-5)
Phase 4: Analysis
- Aggregate results across seeds
- Compute statistical significance (paired t-test, bootstrap CI)
- Generate comparison tables and plots:
python src/plot.py --results runs/ --output paper/figures/ - Run ablation analysis
- Identify surprising findings (investigate, don't hide)
Phase 5: Paper Drafting
Follow venue template structure:
- Abstract — problem, method, key result, impact (write last)
- Introduction — motivation, contribution summary, roadmap
- Related Work — position within literature (from Phase 1)
- Method — approach, architecture, training procedure
- Experiments — setup, main results, ablations, analysis
- Conclusion — summary, limitations, future work
Writing principles:
- One idea per paragraph
- Figures tell the story — design figures first, write text around them
- Tables for comparisons — main results table + ablation table
- Reproducibility — include all hyperparameters, release code
Phase 6: Self-Review & Revision
Use academic-paper-review skill to self-review:
- Read the paper cold (fresh eyes)
- Check methodology soundness, novelty, reproducibility
- Identify weaknesses and fix them
- Get feedback from collaborators
Phase 7: Submission
- Check venue formatting requirements
- Verify page limits
- Anonymize for blind review (if applicable)
- Prepare supplementary material (code, data, extended results)
- Submit before deadline (not at 23:59)
Statistical Analysis
# Multiple seeds — compute mean ± std
python3 -c "
import numpy as np
results = [0.85, 0.83, 0.86, 0.84, 0.82] # per-seed results
print(f'Mean: {np.mean(results):.4f} ± {np.std(results):.4f}')
"
# Paired t-test vs baseline
python3 -c "
from scipy import stats
baseline = [0.80, 0.79, 0.81, 0.78, 0.80]
ours = [0.85, 0.83, 0.86, 0.84, 0.82]
t, p = stats.ttest_rel(ours, baseline)
print(f't={t:.3f}, p={p:.4f}')
"
Pitfalls
- Single seed: results from one seed are not reliable. Use 3-5 minimum.
- Cherry-picking: report all results, not just the best seed.
- No ablations: reviewers will ask "does each component matter?" — answer proactively.
- Missing related work: reviewers know the field. Cite comprehensively.
- Unclear contributions: list contributions explicitly in the introduction.
- Overclaiming: "state-of-the-art" needs evidence across datasets, not one.
- Last-minute submission: servers crash at deadlines. Submit early.
Dependencies
This skill benefits from: numpy, scipy, matplotlib (analysis + plots).
Install via pip install numpy scipy matplotlib.
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |