PaperOrchestra: A Multi-Agent Framework for Automated AI Research Paper Writing

• URL: http://arxiv.org/abs/2604.05018v1
• Date: Mon, 06 Apr 2026 18:00:00 GMT
• Title: PaperOrchestra: A Multi-Agent Framework for Automated AI Research Paper Writing
• Abstract summary: PaperOrchestra is a multi-agent framework for automated AI research paper writing.<n>It transforms unconstrained preconstrained materials into submission-ready manuscripts.<n>PaperWritingBench is the first standardized benchmark of reverse-engineered raw materials from 200 top-tier AI conference papers.
• Score: 43.290959385976414
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Synthesizing unstructured research materials into manuscripts is an essential yet under-explored challenge in AI-driven scientific discovery. Existing autonomous writers are rigidly coupled to specific experimental pipelines, and produce superficial literature reviews. We introduce PaperOrchestra, a multi-agent framework for automated AI research paper writing. It flexibly transforms unconstrained pre-writing materials into submission-ready LaTeX manuscripts, including comprehensive literature synthesis and generated visuals, such as plots and conceptual diagrams. To evaluate performance, we present PaperWritingBench, the first standardized benchmark of reverse-engineered raw materials from 200 top-tier AI conference papers, alongside a comprehensive suite of automated evaluators. In side-by-side human evaluations, PaperOrchestra significantly outperforms autonomous baselines, achieving an absolute win rate margin of 50%-68% in literature review quality, and 14%-38% in overall manuscript quality.

Related papers

• PaperDoctor: Evidence-Grounded and Actionable Feedback for Scientific Papers in Progress [130.58520199343707]
  PaperDoctor is an agent framework for pre-submission feedback with three key innovations.<n>First, a holistic hierarchical framework evaluates writing, layout, references, code, theory, prior work, and experiments.<n>Second, each finding contains an observation, a pointer to specific evidence such as a sentence, equation, or code line, and a revision suggestion.<n>Third, PaperDoctor selectively rebuilds and reruns experiments based on claim importance and compute budget.
  arXiv  Detail & Related papers  (2026-09-15T11:08:11Z)
• Positioning manuscripts in the scientific landscape with agentic AI [21.15576036232353]
  Publishing a research manuscript is a routine yet demanding part of scientific life.<n>Recent advances in large language model (LLM)-based agentic AI have shown promise across a range of scientific tasks.<n>We introduce PASS, an agentic system that understands manuscripts within their domain-specific literature context.
  arXiv  Detail & Related papers  (2026-09-12T07:10:14Z)
• Deep Academic Survey: Stateful Agentic Closed-Loop Paradigm for Academic Survey Automation [60.151050016855834]
  DAS is a stateful agentic framework for generating publication-oriented academic surveys.<n>Its key idea is to separate reusable paper analysis from topic-specific manuscript construction.<n>DAS achieves the highest average in all four dimensions, with an overall score of 4.34 compared with 4.03 for the strongest competitor.
  arXiv  Detail & Related papers  (2026-08-18T17:29:00Z)
• Prompt-to-Paper: Agentic AI System for Bioinformatics [0.0]
  We present Prompt-to-Paper, a multi-agent framework that directly addresses this evaluation gap through three integrated innovations.<n>First, a deterministic retrieval-augmented generation pipeline with section-aware relevance scoring and snowball citation expansion grounds every claim in a verifiable corpus of 60--100 papers.<n>Second, an autonomous coding agent executes real computational biology experiments replacing synthetic outputs with genuine numerical results.<n>Third, an eight-dimensional automated quality scorer, benchmarked with approximate reference statistics from published papers and augmented with explicit hallucination penalties, provides standardized, reproducible quality assessments.
  arXiv  Detail & Related papers  (2026-07-05T21:30:24Z)
• AI for Auto-Research: Roadmap & User Guide [107.0834449839233]
  We present an end-to-end analysis of AI across the complete research lifecycle.<n>We identify a sharp, stage-dependent boundary between reliable assistance and unreliable autonomy.<n>We show that greater automation can obscure rather than eliminate failure modes.
  arXiv  Detail & Related papers  (2026-05-18T17:08:26Z)
• AutoResearchBench: Benchmarking AI Agents on Complex Scientific Literature Discovery [55.70879973230979]
  AutoResearchBench is a benchmark for autonomous scientific literature discovery.<n>Compared to previous benchmarks on agentic web browsing, AutoResearchBench is research-oriented.<n>Even the most powerful LLMs, despite having largely conquered general agentic web-browsing benchmarks such as BrowseComp, achieve only 9.39% accuracy on Deep Research and 9.31% IoU on Wide Research.
  arXiv  Detail & Related papers  (2026-04-28T06:05:17Z)
• AutoSOTA: An End-to-End Automated Research System for State-of-the-Art AI Model Discovery [49.77172027295716]
  AutoSOTA is an end-to-end automated research system that advances the latest State-Of-The-Art (SOTA) models published in top-tier AI papers.<n>It successfully discovers 105 new SOTA models that surpass the original reported methods, averaging approximately five hours per paper.
  arXiv  Detail & Related papers  (2026-04-07T07:52:01Z)
• Paper Reconstruction Evaluation: Evaluating Presentation and Hallucination in AI-written Papers [41.637398542914816]
  This paper introduces the first systematic evaluation framework for quantifying the quality and risks of papers written by modern coding agents.<n>PaperRecon disentangles the evaluation of the AI-written papers into two dimensions, Presentation and Hallucination.<n>For evaluation, we introduce PaperWrite-Bench, a benchmark of 51 papers from top-tier venues across diverse domains published after 2025.
  arXiv  Detail & Related papers  (2026-04-01T16:48:04Z)
• ARISE: Agentic Rubric-Guided Iterative Survey Engine for Automated Scholarly Paper Generation [7.437989615069771]
  ARISE is an agentic-guided Iterative Survey Engine for automated generation and continuous refinement of academic survey papers.<n>ARISE employs a modular architecture composed of specialized large language model agents, each mirroring distinct scholarly roles such as topic expansion, citation curation, literature summarization, manuscript drafting, and peer-review-based evaluation.<n>ARISE consistently surpasses baseline methods across metrics of comprehensiveness, accuracy, formatting, and overall scholarly rigor.
  arXiv  Detail & Related papers  (2025-11-21T14:14:35Z)
• Paper2Web: Let's Make Your Paper Alive! [51.75896846964824]
  We introduce Paper2Web, a benchmark dataset and framework for assessing academic webpage generation.<n>We present PWAgent, an autonomous pipeline that converts scientific papers into interactive and multimedia-rich academic homepages.
  arXiv  Detail & Related papers  (2025-10-17T17:35:58Z)
• LiRA: A Multi-Agent Framework for Reliable and Readable Literature Review Generation [66.09346158850308]
  We present LiRA (Literature Review Agents), a multi-agent collaborative workflow which emulates the human literature review process.<n>LiRA utilizes specialized agents for content outlining, subsection writing, editing, and reviewing, producing cohesive and comprehensive review articles.<n>We evaluate LiRA in real-world scenarios using document retrieval and assess its robustness to reviewer model variation.
  arXiv  Detail & Related papers  (2025-10-01T12:14:28Z)
• AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage [62.049868205196425]
  AutoReproduce is a framework capable of automatically reproducing experiments described in research papers in an end-to-end manner.<n>Results show that AutoReproduce achieves an average performance gap of $22.1%$ on $89.74%$ of the executable experiment runs.
  arXiv  Detail & Related papers  (2025-05-27T03:15:21Z)
• PaperBench: Evaluating AI's Ability to Replicate AI Research [3.4567792239799133]
  PaperBench is a benchmark evaluating the ability of AI agents to replicate state-of-the-art AI research.<n>Agents must replicate 20 ICML 2024 Spotlight and Oral papers from scratch.<n>PaperBench contains 8,316 individually gradable tasks.
  arXiv  Detail & Related papers  (2025-04-02T15:55:24Z)
• The ICML 2023 Ranking Experiment: Examining Author Self-Assessment in ML/AI Peer Review [49.43514488610211]
  Author-provided rankings could be leveraged to improve peer review processes at machine learning conferences.<n>We focus on the Isotonic Mechanism, which calibrates raw review scores using the author-provided rankings.<n>We propose several cautious, low-risk applications of the Isotonic Mechanism and author-provided rankings in peer review.
  arXiv  Detail & Related papers  (2024-08-24T01:51:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.