DECKBench: Benchmarking Multi-Agent Frameworks for Academic Slide Generation and Editing
- URL: http://arxiv.org/abs/2602.13318v1
- Date: Tue, 10 Feb 2026 19:49:06 GMT
- Title: DECKBench: Benchmarking Multi-Agent Frameworks for Academic Slide Generation and Editing
- Abstract summary: We introduce the Deck Edits and Compliance Benchmark (DECKBench), an evaluation framework for multi-agent slide generation and editing.<n>Our evaluation protocol systematically assesses slide-level and deck-level fidelity, coherence, layout quality, and multi-turn instruction following.<n>We further implement a modular multi-agent baseline system that decomposes the slide generation and editing task into paper parsing and summarization, slide planning, HTML creation, and iterative editing.
- Score: 11.47669592304878
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Automatically generating and iteratively editing academic slide decks requires more than document summarization. It demands faithful content selection, coherent slide organization, layout-aware rendering, and robust multi-turn instruction following. However, existing benchmarks and evaluation protocols do not adequately measure these challenges. To address this gap, we introduce the Deck Edits and Compliance Kit Benchmark (DECKBench), an evaluation framework for multi-agent slide generation and editing. DECKBench is built on a curated dataset of paper to slide pairs augmented with realistic, simulated editing instructions. Our evaluation protocol systematically assesses slide-level and deck-level fidelity, coherence, layout quality, and multi-turn instruction following. We further implement a modular multi-agent baseline system that decomposes the slide generation and editing task into paper parsing and summarization, slide planning, HTML creation, and iterative editing. Experimental results demonstrate that the proposed benchmark highlights strengths, exposes failure modes, and provides actionable insights for improving multi-agent slide generation and editing systems. Overall, this work establishes a standardized foundation for reproducible and comparable evaluation of academic presentation generation and editing. Code and data are publicly available at https://github.com/morgan-heisler/DeckBench .
Related papers
- SLIDEFORGE: An LLM Agent for Controllable Editing of Slides as Structured Artifacts [9.707578528201315]
We present SLIDEFORGE, a controllable AI-assisted slide editing framework.<n>It links visual decomposition, native pptx object structure, and perceptual organization.<n>It supports theme-preserving reconstruction through slide-preserving operations and rendered-state verification.
arXiv Detail & Related papers (2026-09-02T19:44:31Z) - OmniEdit-Bench: A Comprehensive Benchmark for Instruction-based Video Editing [53.21868575804758]
We introduce a comprehensive and structured benchmark for instruction-based video editing (IVE)<n>Our benchmark decomposes editing tasks into multiple video-specific dimensions, including spatial, temporal, audio, and reference-based editing.<n>We propose an evaluation framework that assesses editing quality from four complementary dimensions: accuracy, preservation, realism, and consistency.
arXiv Detail & Related papers (2026-08-05T16:58:48Z) - MiLDEdit: Reasoning-Based Multi-Layer Design Document Editing [67.28337411397062]
We introduce the Multi-Layer Document Editing Agent (MiLDEAgent)<n>MiLDEAgent is a reasoning-based framework that combines an RL-trained multimodal reasoner for layer-wise understanding with an image editor for targeted modifications.<n>MiLDEAgent achieves strong layer-aware reasoning and precise editing, significantly outperforming all open-source baselines and attaining performance comparable to closed-source models.
arXiv Detail & Related papers (2026-01-08T04:38:07Z) - SlideGen: Collaborative Multimodal Agents for Scientific Slide Generation [26.4357968329723]
SlideGen is an agentic, modular, and visual in the loop framework for scientific paper to slide generation.<n>It orchestrates a group of vision language agents that reason collaboratively over the document structure and semantics, producing editableX slides with logical flow and compelling visual presentation.
arXiv Detail & Related papers (2025-12-04T07:22:16Z) - PPTArena: A Benchmark for Agentic PowerPoint Editing [57.55409966131844]
We introduce a benchmark for PowerPoint editing that measures reliable modifications to real slides under natural-language instructions.<n>PPTArena focuses on in-place editing across 100 decks, 2125 slides, and over 800 targeted edits covering text, charts, tables, animations, and master-level styles.<n>We propose a structure-aware slide-editing agent that plans semantic edit sequences, routes between high-level programmatic tools, and deterministic XML operations for precise control.
arXiv Detail & Related papers (2025-12-02T18:59:50Z) - ComplexBench-Edit: Benchmarking Complex Instruction-Driven Image Editing via Compositional Dependencies [13.525744033075785]
Real-world scenarios often involve complex, multi-step instructions, particularly chain'' instructions where operations are interdependent.<n>Current models struggle with these intricate directives, and existing benchmarks inadequately evaluate such capabilities.<n>We introduce ComplexBench-Edit, a novel benchmark designed to systematically assess model performance on complex, multi-instruction, and chain-dependent image editing tasks.
arXiv Detail & Related papers (2025-06-15T12:22:55Z) - Image Editing As Programs with Diffusion Models [69.05164729625052]
We introduce Image Editing As Programs (IEAP), a unified image editing framework built upon the Diffusion Transformer (DiT) architecture.<n>IEAP approaches instructional editing through a reductionist lens, decomposing complex editing instructions into sequences of atomic operations.<n>Our framework delivers superior accuracy and semantic fidelity, particularly for complex, multi-step instructions.
arXiv Detail & Related papers (2025-06-04T16:57:24Z) - CompBench: Benchmarking Complex Instruction-guided Image Editing [63.347846732450364]
CompBench is a large-scale benchmark for complex instruction-guided image editing.<n>We propose an MLLM-human collaborative framework with tailored task pipelines.<n>We propose an instruction decoupling strategy that disentangles editing intents into four key dimensions.
arXiv Detail & Related papers (2025-05-18T02:30:52Z) - EditEval: An Instruction-Based Benchmark for Text Improvements [73.5918084416016]
This work presents EditEval: An instruction-based, benchmark and evaluation suite for automatic evaluation of editing capabilities.
We evaluate several pre-trained models, which shows that InstructGPT and PEER perform the best, but that most baselines fall below the supervised SOTA.
Our analysis shows that commonly used metrics for editing tasks do not always correlate well, and that optimization for prompts with the highest performance does not necessarily entail the strongest robustness to different models.
arXiv Detail & Related papers (2022-09-27T12:26:05Z) - DOC2PPT: Automatic Presentation Slides Generation from Scientific
Documents [76.19748112897177]
We present a novel task and approach for document-to-slide generation.
We propose a hierarchical sequence-to-sequence approach to tackle our task in an end-to-end manner.
Our approach exploits the inherent structures within documents and slides and incorporates paraphrasing and layout prediction modules to generate slides.
arXiv Detail & Related papers (2021-01-28T03:21:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.