Ar9av/PaperOrchestra / skills/agent-research-aggregator
agent-research-aggregator
VerifiedPre-pipeline aggregator that scans AI agent cache directories (.claude, .cursor, .antigravity, .openclaw) or any user-specified directory for experimentation logs, extracts insights and numeric results, and formats them as PaperOrchestra-ready inputs (idea.md + experimental_log.md). TRIGGER when the user says "aggregate my agent logs for paper writing", "extract experiments from my coding agent history", "prepare PaperOrchestra inputs from my cache", "turn my agent logs into a paper", mentions a folder or directory they want to use as the basis for a paper, or wants to run PaperOrchestra but only has scattered agent experiment histories rather than structured inputs. Run this BEFORE paper-orchestra. Also called automatically by paper-orchestra when workspace/inputs/idea.md or workspace/inputs/experimental_log.md are missing.
| File | Kind | Size |
|---|---|---|
| SKILL.md | skill md | 14.1 KB |
| references/extraction-prompt.md | markdown | 4.5 KB |
| references/log-formats.md | markdown | 5.5 KB |
| references/synthesis-prompt.md | markdown | 5.1 KB |
| scripts/discover_logs.py | script | 15.7 KB |
| scripts/extract_experiments.py | script | 7.9 KB |
| scripts/format_po_inputs.py | script | 13.4 KB |
scripts/format_po_inputs.py
View on GitHub#!/usr/bin/env python3
"""
format_po_inputs.py — Phase 4 of agent-research-aggregator.
Converts synthesis.json (Phase 3 output) into PaperOrchestra-compatible input
files: idea.md (Sparse variant) and experimental_log.md.
Optionally writes an aggregation_report.md audit trail.
Usage:
python format_po_inputs.py \\
--synthesis workspace/ara/synthesis.json \\
--out workspace/inputs/ \\
--report workspace/ara/aggregation_report.md
# Dry-run (print to stdout, don't write files):
python format_po_inputs.py \\
--synthesis workspace/ara/synthesis.json \\
--out workspace/inputs/ \\
--dry-run
"""
import argparse
import json
import sys
from datetime import datetime, timezone
from pathlib import Path
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def load_synthesis(path: str) -> dict:
p = Path(path)
if not p.exists():
print(f"[ERROR] synthesis.json not found: {path}", file=sys.stderr)
sys.exit(1)
with p.open(encoding="utf-8") as f:
data = json.load(f)
return data
def rows_to_markdown_table(headers: list, rows: list) -> str:
"""Convert headers + rows to a GitHub-Flavored Markdown table."""
col_widths = [len(str(h)) for h in headers]
for row in rows:
for i, cell in enumerate(row):
if i < len(col_widths):
col_widths[i] = max(col_widths[i], len(str(cell)))
def fmt_row(cells):
parts = []
for i, cell in enumerate(cells):
w = col_widths[i] if i < len(col_widths) else len(str(cell))
parts.append(str(cell).ljust(w))
return "| " + " | ".join(parts) + " |"
separator = "| " + " | ".join("-" * w for w in col_widths) + " |"
lines = [fmt_row(headers), separator]
for row in rows:
lines.append(fmt_row(row))
return "\n".join(lines)
def pluralise(n: int, singular: str, plural: str | None = None) -> str:
if plural is None:
plural = singular + "s"
return f"{n} {singular if n == 1 else plural}"
# ---------------------------------------------------------------------------
# idea.md (PaperOrchestra Sparse variant)
# ---------------------------------------------------------------------------
def build_idea_md(s: dict) -> str:
lines = []
# Derive a title from research_question (first sentence, capitalised)
rq = s.get("research_question", "").strip()
title = rq.split(".")[0].strip() if rq else "Synthesized Research"
lines.append(f"# {title}")
lines.append("")
# Problem section
lines.append("## Problem")
lines.append("")
lines.append(rq if rq else "_[Research question not detected in logs]_")
lines.append("")
# Hypothesis
hyp = s.get("hypothesis", "").strip()
lines.append("## Hypothesis")
lines.append("")
lines.append(hyp if hyp else "_[Hypothesis not detected in logs]_")
lines.append("")
# Method
method = s.get("method_summary", "").strip()
lines.append("## Method")
lines.append("")
lines.append(method if method else "_[Method not detected in logs]_")
lines.append("")
# Key contributions
contributions = s.get("key_contributions", [])
lines.append("## Key Contributions")
lines.append("")
if contributions:
for c in contributions:
lines.append(f"- {c}")
else:
lines.append("_[Contributions not detected in logs]_")
lines.append("")
# Open questions (optional)
open_qs = s.get("open_questions", [])
if open_qs:
lines.append("## Open Questions")
lines.append("")
for q in open_qs:
lines.append(f"- {q}")
lines.append("")
return "\n".join(lines)
# ---------------------------------------------------------------------------
# experimental_log.md (PaperOrchestra Experimental Log format)
# ---------------------------------------------------------------------------
def build_experimental_log_md(s: dict) -> str:
lines = []
lines.append("# Experimental Log")
lines.append("")
lines.append(
"_Generated by agent-research-aggregator from AI agent cache logs._"
)
lines.append("")
# Section 1: Experimental Setup
lines.append("## 1. Experimental Setup")
lines.append("")
setup = s.get("experimental_setup", {})
datasets = setup.get("datasets", [])
if datasets:
lines.append("**Datasets:**")
for d in datasets:
lines.append(f"- {d}")
lines.append("")
baselines = setup.get("baselines", [])
if baselines:
lines.append("**Baselines:**")
for b in baselines:
lines.append(f"- {b}")
lines.append("")
metrics = setup.get("metrics", [])
if metrics:
lines.append("**Metrics:**")
for m in metrics:
lines.append(f"- {m}")
lines.append("")
impl = setup.get("implementation", "").strip()
if impl:
lines.append("**Implementation:**")
lines.append("")
lines.append(impl)
lines.append("")
notes = setup.get("notes", "").strip()
if notes:
lines.append("**Notes:**")
lines.append("")
lines.append(notes)
lines.append("")
if not any([datasets, baselines, metrics, impl, notes]):
lines.append("_[Experimental setup not detected in logs]_")
lines.append("")
# Section 2: Raw Numeric Data
lines.append("## 2. Raw Numeric Data")
lines.append("")
tables = s.get("results_tables", [])
if tables:
for table in tables:
title = table.get("title", "Results")
headers = table.get("headers", [])
rows = table.get("rows", [])
confidence = table.get("confidence", "")
lines.append(f"### {title}")
if confidence in ("low", "medium"):
lines.append(f"_[Data confidence: {confidence}]_")
lines.append("")
if headers and rows:
lines.append(rows_to_markdown_table(headers, rows))
elif headers:
lines.append(rows_to_markdown_table(headers, []))
lines.append("_[No rows found in logs]_")
else:
lines.append("_[Table structure not recovered]_")
lines.append("")
else:
lines.append(
"_[No numeric result tables detected in logs. "
"PaperOrchestra's section-writing agent needs numeric data — "
"consider adding result tables manually before running paper-orchestra.]_"
)
lines.append("")
# Section 3: Qualitative Observations
lines.append("## 3. Qualitative Observations")
lines.append("")
qual = s.get("qualitative_observations", "").strip()
if qual:
lines.append(qual)
lines.append("")
else:
lines.append("_[Qualitative observations not detected in logs]_")
lines.append("")
# Iteration History subsection (if present)
iterations = s.get("iteration_history", [])
if iterations:
lines.append("### Iteration History")
lines.append("")
for it in iterations:
iter_id = it.get("iteration_id", "")
desc = it.get("description", "")
outcome = it.get("outcome", "")
lines.append(f"**{iter_id}:** {desc}")
if outcome:
lines.append(f" - Outcome: {outcome}")
lines.append("")
return "\n".join(lines)
# ---------------------------------------------------------------------------
# Aggregation report
# ---------------------------------------------------------------------------
def build_report_md(s: dict, idea_path: Path, log_path: Path,
discovered_path: Path | None) -> str:
now = datetime.now(tz=timezone.utc).isoformat()
lines = []
lines.append("# Aggregation Report")
lines.append("")
lines.append(f"_Generated at: {now}_")
lines.append("")
# Files written
lines.append("## Files Written")
lines.append("")
lines.append(f"- `{idea_path}` ({idea_path.stat().st_size if idea_path.exists() else '?'} bytes)")
lines.append(f"- `{log_path}` ({log_path.stat().st_size if log_path.exists() else '?'} bytes)")
lines.append("")
# Synthesis summary
lines.append("## Synthesis Summary")
lines.append("")
tables = s.get("results_tables", [])
iters = s.get("iteration_history", [])
open_qs = s.get("open_questions", [])
warnings = s.get("data_quality_warnings", [])
lines.append(f"- Research question detected: {'yes' if s.get('research_question') else 'no'}")
lines.append(f"- Hypothesis detected: {'yes' if s.get('hypothesis') else 'no'}")
lines.append(f"- Result tables: {len(tables)}")
lines.append(f"- Iterations detected: {len(iters)}")
lines.append(f"- Open questions: {len(open_qs)}")
lines.append(f"- Conflicting research questions: {s.get('research_question_count', 1)}")
lines.append("")
# Data quality warnings
if warnings:
lines.append("## Data Quality Warnings")
lines.append("")
for w in warnings:
lines.append(f"- {w}")
lines.append("")
lines.append(
"_Review these warnings before running paper-orchestra. "
"Tables with `[UNVERIFIED]` data may cause the section-writing "
"agent to produce inaccurate numeric claims._"
)
lines.append("")
# Table confidence summary
if tables:
lines.append("## Result Table Confidence")
lines.append("")
for t in tables:
conf = t.get("confidence", "unknown")
src = ", ".join(t.get("source_experiment_ids", []))
lines.append(f"- **{t.get('title', 'Untitled')}**: {conf} (sources: {src or 'unknown'})")
lines.append("")
# Missing required PaperOrchestra inputs
lines.append("## Next Steps for PaperOrchestra")
lines.append("")
lines.append("The following files are still required before running `paper-orchestra`:")
lines.append("")
lines.append("| File | Status |")
lines.append("|---|---|")
lines.append(f"| `workspace/inputs/idea.md` | ✓ generated (review recommended) |")
lines.append(f"| `workspace/inputs/experimental_log.md` | ✓ generated (review recommended) |")
lines.append(f"| `workspace/inputs/template.tex` | **MISSING — ask user to provide** |")
lines.append(f"| `workspace/inputs/conference_guidelines.md` | **MISSING — ask user to provide** |")
lines.append("")
return "\n".join(lines)
# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------
def main():
parser = argparse.ArgumentParser(description="Format PaperOrchestra inputs from synthesis.json")
parser.add_argument("--synthesis", required=True, help="Path to synthesis.json")
parser.add_argument("--out", required=True, help="Output directory for idea.md + experimental_log.md")
parser.add_argument("--report", default=None, help="Optional path for aggregation_report.md")
parser.add_argument("--dry-run", action="store_true",
help="Print generated files to stdout instead of writing")
parser.add_argument("--discovered", default=None,
help="Optional path to discovered_logs.json for richer report")
args = parser.parse_args()
s = load_synthesis(args.synthesis)
# Warn if multiple research questions
rq_count = s.get("research_question_count", 1)
if rq_count > 1:
print(
f"[WARN] synthesis.json indicates {rq_count} distinct research questions.",
file=sys.stderr,
)
print(
"[WARN] The generated idea.md will contain all of them. "
"Ask the user which to target before running paper-orchestra.",
file=sys.stderr,
)
idea_content = build_idea_md(s)
log_content = build_experimental_log_md(s)
out_dir = Path(args.out)
if args.dry_run:
print("=== workspace/inputs/idea.md ===")
print(idea_content)
print("\n=== workspace/inputs/experimental_log.md ===")
print(log_content)
return
out_dir.mkdir(parents=True, exist_ok=True)
idea_path = out_dir / "idea.md"
log_path = out_dir / "experimental_log.md"
with idea_path.open("w", encoding="utf-8") as f:
f.write(idea_content)
print(f"Written: {idea_path}")
with log_path.open("w", encoding="utf-8") as f:
f.write(log_content)
print(f"Written: {log_path}")
# Report
if args.report:
discovered_path = Path(args.discovered) if args.discovered else None
report_content = build_report_md(s, idea_path, log_path, discovered_path)
report_path = Path(args.report)
report_path.parent.mkdir(parents=True, exist_ok=True)
with report_path.open("w", encoding="utf-8") as f:
f.write(report_content)
print(f"Written: {report_path}")
# Final check: warn if no results tables
if not s.get("results_tables"):
print(
"\n[WARN] No result tables were synthesized. "
"PaperOrchestra's section-writing agent requires numeric data — "
"consider adding result tables to experimental_log.md manually.",
file=sys.stderr,
)
print("\nDone. Review idea.md and experimental_log.md with the user before")
print("running paper-orchestra.")
if __name__ == "__main__":
main()
Related skills
| Skill | Stars | Updated | Works with | Status | Install |
|---|---|---|---|---|---|
| writing-skillsobra/superpowersUse when creating new skills, editing existing skills, or verifying skills work before deployment | Stars: 322k | Updated | Claude CodeCodex | Verified | Install |
| xlsxanthropics/skillsUse this skill any time a spreadsheet file is the primary input or output. This means any task where the user wants to: open, read, edit, or fix an existing .xlsx, .xlsm, .xltx, .csv, or .tsv file (e.g., adding columns, computing formulas, formatting, charting, cleaning messy data); create a new spreadsheet from scratch or from other data sources; or convert between tabular file formats. Trigger especially when the user references a spreadsheet file by name or path — even casually (like "the xlsx in my downloads") — and wants something done to it or produced from it. Also trigger for cleaning or restructuring messy tabular data files (malformed rows, misplaced headers, junk data) into proper spreadsheets. The deliverable must be a spreadsheet file. Do NOT trigger when the primary deliverable is a Word document, HTML report, standalone Python script, database pipeline, or Google Sheets API integration, even if tabular data is involved. | Stars: 198k | Updated | Claude CodeCodex | Verified | Install |
| shipping-and-launchaddyosmani/agent-skillsPrepares production launches. Use when preparing to deploy to production, or when asking what needs to be in place before shipping. Use when you need a pre-launch checklist, when setting up monitoring, when planning a staged rollout, or when you need a rollback strategy. | Stars: 119k | Updated | Claude CodeCodex | Verified | Install |
| childhoodthedotmack/claude-memBabysit a shipped release after merge, publication, or deployment. Verify published packages, production behavior, installed plugins, and late CI or user reports until the observation window is clean; use babysit for PR readiness before release. | Stars: 118k | Updated | Claude CodeCodex | Verified | Install |