Paper2SysArch: Structure-Constrained System Architecture Generation from Scientific Papers

• URL: http://arxiv.org/abs/2511.18036v1
• Date: Sat, 22 Nov 2025 12:24:30 GMT
• Title: Paper2SysArch: Structure-Constrained System Architecture Generation from Scientific Papers
• Authors: Ziyi Guo, Zhou Liu, Wentao Zhang,
• Abstract summary: We introduce a novel benchmark to quantitatively evaluate the automated generation of diagrams from text.<n>It consists of 3,000 research papers paired with their corresponding high-quality ground-truth diagrams and is accompanied by a three-tiered evaluation metric.<n>We propose Paper2Arch, an end-to-end system that leverages multi-agent collaboration to convert papers into structured, editable diagrams.
• Score: 10.395280181257737
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The manual creation of system architecture diagrams for scientific papers is a time-consuming and subjective process, while existing generative models lack the necessary structural control and semantic understanding for this task. A primary obstacle hindering research and development in this domain has been the profound lack of a standardized benchmark to quantitatively evaluate the automated generation of diagrams from text. To address this critical gap, we introduce a novel and comprehensive benchmark, the first of its kind, designed to catalyze progress in automated scientific visualization. It consists of 3,000 research papers paired with their corresponding high-quality ground-truth diagrams and is accompanied by a three-tiered evaluation metric assessing semantic accuracy, layout coherence, and visual quality. Furthermore, to establish a strong baseline on this new benchmark, we propose Paper2SysArch, an end-to-end system that leverages multi-agent collaboration to convert papers into structured, editable diagrams. To validate its performance on complex cases, the system was evaluated on a manually curated and more challenging subset of these papers, where it achieves a composite score of 69.0. This work's principal contribution is the establishment of a large-scale, foundational benchmark to enable reproducible research and fair comparison. Meanwhile, our proposed system serves as a viable proof-of-concept, demonstrating a promising path forward for this complex task.

Related papers

• SciFig: Towards Automating Scientific Figure Generation [41.73701976318102]
  SciFig is an end-to-end AI agent system that generates publication-ready pipeline figures directly from research paper texts.<n>We introduce a rubric-based evaluation framework that analyzes 2,219 real scientific figures to extract evaluation rubrics.<n>SciFig demonstrates remarkable performance: achieving 70.1$%$ overall quality on dataset-level evaluation and 66.2$%$ on paper-specific evaluation.
  arXiv  Detail & Related papers  (2026-01-07T20:56:58Z)
• DeepSynth-Eval: Objectively Evaluating Information Consolidation in Deep Survey Writing [53.85037373860246]
  We introduce Deep Synth-Eval, a benchmark designed to objectively evaluate information consolidation capabilities.<n>We propose a fine-grained evaluation protocol using General Checklists (for factual coverage) and Constraint Checklists (for structural organization)<n>Our results demonstrate that agentic plan-and-write significantly outperform single-turn generation.
  arXiv  Detail & Related papers  (2026-01-07T03:07:52Z)
• Enhancing Automated Paper Reproduction via Prompt-Free Collaborative Agents [8.185402940269794]
  We propose a prompt-free collaborative agent framework that automatically enhances the quality of paper-to-code generation.<n>Our approach employs two collaborative agents: a verification agent that examines whether the outputs at each step satisfy the requirements specified in the corresponding system prompt, and a refinement agent that revises the outputs based on the identified issues.
  arXiv  Detail & Related papers  (2025-12-02T14:24:23Z)
• Deep Literature Survey Automation with an Iterative Workflow [30.923568155892184]
  ours is a framework based on recurrent outline generation to ensure both exploration and coherence.<n>To provide faithful paper-level grounding, we design paper cards that distill each paper into its contributions, methods, and findings.<n>Experiments on both established and emerging topics show that ours substantially outperforms state-of-the-art baselines in content coverage, structural coherence, and citation quality.
  arXiv  Detail & Related papers  (2025-10-24T14:41:26Z)
• Unifying Tree Search Algorithm and Reward Design for LLM Reasoning: A Survey [92.71325249013535]
  Deliberative tree search is a cornerstone of Large Language Model (LLM) research.<n>This paper introduces a unified framework that deconstructs search algorithms into three core components.
  arXiv  Detail & Related papers  (2025-10-11T03:29:18Z)
• Context-Aware Hierarchical Taxonomy Generation for Scientific Papers via LLM-Guided Multi-Aspect Clustering [59.54662810933882]
  Existing taxonomy construction methods, leveraging unsupervised clustering or direct prompting of large language models, often lack coherence and granularity.<n>We propose a novel context-aware hierarchical taxonomy generation framework that integrates LLM-guided multi-aspect encoding with dynamic clustering.
  arXiv  Detail & Related papers  (2025-09-23T15:12:58Z)
• ResearchPulse: Building Method-Experiment Chains through Multi-Document Scientific Inference [19.363580418167114]
  We formalize multi-document scientific inference, a new task that extracts and aligns motivation, methodology, and experimental results across related papers.<n>We present ResearchPulse, an agent-based framework that integrates instruction planning, scientific content extraction, and structured visualization.<n>To support this task, we introduce ResearchPulse-Bench, a citation-aware benchmark of annotated paper clusters.
  arXiv  Detail & Related papers  (2025-09-03T15:45:03Z)
• PosterForest: Hierarchical Multi-Agent Collaboration for Scientific Poster Generation [28.02969134846803]
  We introduce the textitPoster Tree, a hierarchical intermediate representation that jointly encodes document structure and visual-textual relationships.<n>Our framework employs a multi-agent collaboration strategy, where agents specializing in content summarization and layout planning iteratively coordinate and provide mutual feedback.
  arXiv  Detail & Related papers  (2025-08-29T15:36:06Z)
• DREAM: Document Reconstruction via End-to-end Autoregressive Model [53.51754520966657]
  We present an innovative autoregressive model specifically designed for document reconstruction, referred to as Document Reconstruction via End-to-end Autoregressive Model (DREAM)<n>We establish a standardized definition of the document reconstruction task, and introduce a novel Document Similarity Metric (DSM) and DocRec1K dataset for assessing the performance of the task.
  arXiv  Detail & Related papers  (2025-07-08T09:24:07Z)
• Topological Structure Learning Should Be A Research Priority for LLM-Based Multi-Agent Systems [69.95482609893236]
  Large Language Model-based Multi-Agent Systems (MASs) have emerged as a powerful paradigm for tackling complex tasks through collaborative intelligence.<n>We call for a paradigm shift toward emphtopology-aware MASs that explicitly model and dynamically optimize the structure of inter-agent interactions.
  arXiv  Detail & Related papers  (2025-05-28T15:20:09Z)
• Creativity in LLM-based Multi-Agent Systems: A Survey [56.25583236738877]
  Large language model (LLM)-driven multi-agent systems (MAS) are transforming how humans and AIs collaboratively generate ideas and artifacts.<n>This is the first survey dedicated to creativity in MAS.<n>We focus on text and image generation tasks, and present: (1) a taxonomy of agent proactivity and persona design; (2) an overview of generation techniques, including divergent exploration, iterative refinement, and collaborative synthesis, as well as relevant datasets and evaluation metrics; and (3) a discussion of key challenges, such as inconsistent evaluation standards, insufficient bias mitigation, coordination conflicts, and the lack of unified benchmarks.
  arXiv  Detail & Related papers  (2025-05-27T12:36:14Z)
• XtraGPT: Context-Aware and Controllable Academic Paper Revision [43.263488839387584]
  We propose a human-AI collaboration framework for academic paper revision centered on criteria-guided intent alignment and context-aware modeling.<n>We instantiate the framework in XtraGPT, the first suite of open-source LLMs for context-aware, instruction-guided writing assistance.
  arXiv  Detail & Related papers  (2025-05-16T15:02:19Z)
• Rethinking Word-Level Auto-Completion in Computer-Aided Translation [76.34184928621477]
  Word-Level Auto-Completion (WLAC) plays a crucial role in Computer-Assisted Translation. It aims at providing word-level auto-completion suggestions for human translators. We introduce a measurable criterion to answer this question and discover that existing WLAC models often fail to meet this criterion. We propose an effective approach to enhance WLAC performance by promoting adherence to the criterion.
  arXiv  Detail & Related papers  (2023-10-23T03:11:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.