PaperVoyager : Building Interactive Web with Visual Language Models

• URL: http://arxiv.org/abs/2603.22999v2
• Date: Sat, 28 Mar 2026 06:16:28 GMT
• Title: PaperVoyager : Building Interactive Web with Visual Language Models
• Abstract summary: We propose a Paper-to-Interactive-System Agent that converts research papers into executable interactive web systems.<n>Given a PDF paper, the agent performs end-to-end processing without human intervention.<n>Experiments show that PaperVoyager significantly improves the quality of generated interactive systems.
• Score: 35.73710997735284
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in visual language models have enabled autonomous agents for complex reasoning, tool use, and document understanding. However, existing document agents mainly transform papers into static artifacts such as summaries, webpages, or slides, which are insufficient for technical papers involving dynamic mechanisms and state transitions. In this work, we propose a Paper-to-Interactive-System Agent that converts research papers into executable interactive web systems. Given a PDF paper, the agent performs end-to-end processing without human intervention, including paper understanding, system modeling, and interactive webpage synthesis, enabling users to manipulate inputs and observe dynamic behaviors. To evaluate this task, we introduce a benchmark of 19 research papers paired with expert-built interactive systems as ground truth. We further propose PaperVoyager, a structured generation framework that explicitly models mechanisms and interaction logic during synthesis. Experiments show that PaperVoyager significantly improves the quality of generated interactive systems, offering a new paradigm for interactive scientific paper understanding.

Related papers

• I-WebGenBench : Evaluating Interactivity in LLM-Generated Scientific Web Applications [33.31056974240273]
  We propose a Paper-to-Interactive-System Agent that converts research papers into executable interactive web systems.<n>Given a PDF paper, the agent performs end-to-end processing without human intervention.<n>Experiments show that PaperVoyager significantly improves the quality of generated interactive systems.
  arXiv  Detail & Related papers  (2026-05-30T14:34:02Z)
• DocOS: Towards Proactive Document-Guided Actions in GUI Agents [54.27655693145045]
  textbfDocOS is a benchmark designed to assess document-guided problem solving in fully interactive environments.<n>Experiments reveal that agents struggle to reliably locate relevant information during proactive search and frequently fail to faithfully ground retrieved instructions into precise actions.
  arXiv  Detail & Related papers  (2026-05-18T08:36:04Z)
• ViviDoc: Generating Interactive Documents through Human-Agent Collaboration [6.761074932523358]
  Interactive documents help readers engage with complex ideas through dynamic visualization, interactive animations, and exploratory interfaces.<n>Recent Large Language Model (LLM)-based agents can automate content creation, but directly applying them to interactive document generation often produces outputs that are difficult to control.<n>We present ViviDoc, to the best of our knowledge the first work to systematically address interactive document generation.
  arXiv  Detail & Related papers  (2026-03-30T03:29:32Z)
• Doc To The Future: Infomorphs for Interactive, Multimodal Document Transformation and Generation [4.665470854146965]
  We introduce the notion of "infomorphs" which are modular, user-steerable, AI-augmented transformations that support controlled synthesis, and restructuring of information across formats and modalities.<n>We propose a design space that leverage infomorph-driven document creation by combining Generative AI techniques with user intent and desired information context.<n>DocuCraft allows users to chain together infomorphs that perform operations such as page extraction, content summarization, reformatting, and generation, leveraging Generative AI at each stage to support rich, cross-document and cross-modal transformations.
  arXiv  Detail & Related papers  (2025-12-14T16:25:19Z)
• Paper2Web: Let's Make Your Paper Alive! [51.75896846964824]
  We introduce Paper2Web, a benchmark dataset and framework for assessing academic webpage generation.<n>We present PWAgent, an autonomous pipeline that converts scientific papers into interactive and multimedia-rich academic homepages.
  arXiv  Detail & Related papers  (2025-10-17T17:35:58Z)
• Paper2Agent: Reimagining Research Papers As Interactive and Reliable AI Agents [24.762629506054125]
  We introduce Paper2Agent, an automated framework that converts research papers into AI agents.<n>Paper2Agent transforms research output from passive artifacts into active systems that can accelerate downstream use, adoption, and discovery.
  arXiv  Detail & Related papers  (2025-09-08T17:28:42Z)
• Agentic Web: Weaving the Next Web with AI Agents [109.13815627467514]
  The emergence of AI agents powered by large language models (LLMs) marks a pivotal shift toward the Agentic Web.<n>In this paradigm, agents interact directly with one another to plan, coordinate, and execute complex tasks on behalf of users.<n>We present a structured framework for understanding and building the Agentic Web.
  arXiv  Detail & Related papers  (2025-07-28T17:58:12Z)
• Drama Engine: A Framework for Narrative Agents [0.0]
  Drama Engine is a framework for agentic interaction with large language models designed for narrative purposes. The framework adapts multi-agent system principles to create dynamic, context-aware companions. The framework's applications include multi-agent chats and virtual co-workers for creative writing.
  arXiv  Detail & Related papers  (2024-08-21T12:29:38Z)
• Manual-Guided Dialogue for Flexible Conversational Agents [84.46598430403886]
  How to build and use dialogue data efficiently, and how to deploy models in different domains at scale can be critical issues in building a task-oriented dialogue system. We propose a novel manual-guided dialogue scheme, where the agent learns the tasks from both dialogue and manuals. Our proposed scheme reduces the dependence of dialogue models on fine-grained domain ontology, and makes them more flexible to adapt to various domains.
  arXiv  Detail & Related papers  (2022-08-16T08:21:12Z)
• VIRT: Improving Representation-based Models for Text Matching through Virtual Interaction [50.986371459817256]
  We propose a novel textitVirtual InteRacTion mechanism, termed as VIRT, to enable full and deep interaction modeling in representation-based models. VIRT asks representation-based encoders to conduct virtual interactions to mimic the behaviors as interaction-based models do.
  arXiv  Detail & Related papers  (2021-12-08T09:49:28Z)
• iFacetSum: Coreference-based Interactive Faceted Summarization for Multi-Document Exploration [63.272359227081836]
  iFacetSum integrates interactive summarization together with faceted search. Fine-grained facets are automatically produced based on cross-document coreference pipelines.
  arXiv  Detail & Related papers  (2021-09-23T20:01:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.