ViviDoc: Generating Interactive Documents through Human-Agent Collaboration
- URL: http://arxiv.org/abs/2603.27991v1
- Date: Mon, 30 Mar 2026 03:29:32 GMT
- Title: ViviDoc: Generating Interactive Documents through Human-Agent Collaboration
- Abstract summary: Interactive documents help readers engage with complex ideas through dynamic visualization, interactive animations, and exploratory interfaces.<n>Recent Large Language Model (LLM)-based agents can automate content creation, but directly applying them to interactive document generation often produces outputs that are difficult to control.<n>We present ViviDoc, to the best of our knowledge the first work to systematically address interactive document generation.
- Score: 6.761074932523358
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Interactive documents help readers engage with complex ideas through dynamic visualization, interactive animations, and exploratory interfaces. However, creating such documents remains costly, as it requires both domain expertise and web development skills. Recent Large Language Model (LLM)-based agents can automate content creation, but directly applying them to interactive document generation often produces outputs that are difficult to control. To address this, we present ViviDoc, to the best of our knowledge the first work to systematically address interactive document generation. ViviDoc introduces a multi-agent pipeline (Planner, Styler, Executor, Evaluator). To make the generation process controllable, we provide three levels of human control: (1) the Document Specification (DocSpec) with SRTC Interaction Specifications (State, Render, Transition, Constraint) for structured planning, (2) a content-aware Style Palette for customizing writing and interaction styles, and (3) chat-based editing for iterative refinement. We also construct ViviBench, a benchmark of 101 topics derived from real-world interactive documents across 11 domains, along with a taxonomy of 8 interaction types and a 4-dimensional automated evaluation framework validated against human ratings (Pearson r > 0.84). Experiments show that ViviDoc achieves the highest content richness and interaction quality in both automated and human evaluation. A 12-person user study confirms that the system is easy to use, provides effective control over the generation process, and produces documents that satisfy users.
Related papers
- I-WebGenBench : Evaluating Interactivity in LLM-Generated Scientific Web Applications [33.31056974240273]
We propose a Paper-to-Interactive-System Agent that converts research papers into executable interactive web systems.<n>Given a PDF paper, the agent performs end-to-end processing without human intervention.<n>Experiments show that PaperVoyager significantly improves the quality of generated interactive systems.
arXiv Detail & Related papers (2026-05-30T14:34:02Z) - DocOS: Towards Proactive Document-Guided Actions in GUI Agents [54.27655693145045]
textbfDocOS is a benchmark designed to assess document-guided problem solving in fully interactive environments.<n>Experiments reveal that agents struggle to reliably locate relevant information during proactive search and frequently fail to faithfully ground retrieved instructions into precise actions.
arXiv Detail & Related papers (2026-05-18T08:36:04Z) - PaperVoyager : Building Interactive Web with Visual Language Models [35.73710997735284]
We propose a Paper-to-Interactive-System Agent that converts research papers into executable interactive web systems.<n>Given a PDF paper, the agent performs end-to-end processing without human intervention.<n>Experiments show that PaperVoyager significantly improves the quality of generated interactive systems.
arXiv Detail & Related papers (2026-03-24T09:42:27Z) - Demonstrating ViviDoc: Generating Interactive Documents through Human-Agent Collaboration [4.751545995185441]
We present ViviDoc, a human-agent collaborative system that generates interactive educational documents from a single topic input.<n>ViviDoc introduces a multi-agent pipeline (Planner, Executor, Evaluator) and the Document Specification (DocSpec), a human-readable intermediate representation.<n>Expert evaluation and a user study show that ViviDoc substantially outperforms naive agentic generation and provides an intuitive editing experience.
arXiv Detail & Related papers (2026-03-02T14:27:49Z) - Doc To The Future: Infomorphs for Interactive, Multimodal Document Transformation and Generation [4.665470854146965]
We introduce the notion of "infomorphs" which are modular, user-steerable, AI-augmented transformations that support controlled synthesis, and restructuring of information across formats and modalities.<n>We propose a design space that leverage infomorph-driven document creation by combining Generative AI techniques with user intent and desired information context.<n>DocuCraft allows users to chain together infomorphs that perform operations such as page extraction, content summarization, reformatting, and generation, leveraging Generative AI at each stage to support rich, cross-document and cross-modal transformations.
arXiv Detail & Related papers (2025-12-14T16:25:19Z) - Contextual Document Embeddings [77.22328616983417]
We propose two complementary methods for contextualized document embeddings.
First, an alternative contrastive learning objective that explicitly incorporates the document neighbors into the intra-batch contextual loss.
Second, a new contextual architecture that explicitly encodes neighbor document information into the encoded representation.
arXiv Detail & Related papers (2024-10-03T14:33:34Z) - DocGenome: An Open Large-scale Scientific Document Benchmark for Training and Testing Multi-modal Large Language Models [63.466265039007816]
We present DocGenome, a structured document benchmark constructed by annotating 500K scientific documents from 153 disciplines in the arXiv open-access community.
We conduct extensive experiments to demonstrate the advantages of DocGenome and objectively evaluate the performance of large models on our benchmark.
arXiv Detail & Related papers (2024-06-17T15:13:52Z) - DocSynthv2: A Practical Autoregressive Modeling for Document Generation [43.84027661517748]
This paper proposes a novel approach called Doc Synthv2 through the development of a simple yet effective autoregressive structured model.
Our model, distinct in its integration of both layout and textual cues, marks a step beyond existing layout-generation approaches.
arXiv Detail & Related papers (2024-06-12T16:00:16Z) - Doc2Bot: Accessing Heterogeneous Documents via Conversational Bots [103.54897676954091]
Doc2Bot is a dataset for building machines that help users seek information via conversations.
Our dataset contains over 100,000 turns based on Chinese documents from five domains.
arXiv Detail & Related papers (2022-10-20T07:33:05Z) - Manual-Guided Dialogue for Flexible Conversational Agents [84.46598430403886]
How to build and use dialogue data efficiently, and how to deploy models in different domains at scale can be critical issues in building a task-oriented dialogue system.
We propose a novel manual-guided dialogue scheme, where the agent learns the tasks from both dialogue and manuals.
Our proposed scheme reduces the dependence of dialogue models on fine-grained domain ontology, and makes them more flexible to adapt to various domains.
arXiv Detail & Related papers (2022-08-16T08:21:12Z) - Unified Pretraining Framework for Document Understanding [52.224359498792836]
We present UDoc, a new unified pretraining framework for document understanding.
UDoc is designed to support most document understanding tasks, extending the Transformer to take multimodal embeddings as input.
An important feature of UDoc is that it learns a generic representation by making use of three self-supervised losses.
arXiv Detail & Related papers (2022-04-22T21:47:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.