VLM-SlideEval: Evaluating VLMs on Structured Comprehension and Perturbation Sensitivity in PPT

• URL: http://arxiv.org/abs/2510.22045v1
• Date: Fri, 24 Oct 2025 22:06:56 GMT
• Title: VLM-SlideEval: Evaluating VLMs on Structured Comprehension and Perturbation Sensitivity in PPT
• Abstract summary: Vision-language models (VLMs) are increasingly used to evaluate multimodal content, including presentation slides, but their slide-specific understanding remains underexplored.<n>We introduce VLM-SlideEval, an evaluation framework that probes VLMs along three axes: (1) element-level extraction from slide images aligned to ground truth; (2) to controlled perturbations in geometry, style, and text; and (3) higher-level comprehension, such as recovering a deck's narrative order from shuffled slides.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language models (VLMs) are increasingly used to evaluate multimodal content, including presentation slides, yet their slide-specific understanding remains underexplored {despite their growing role as critics in agentic, model-forward pipelines}. We introduce VLM-SlideEval, an evaluation framework that probes VLMs along three axes: (1) element-level extraction from slide images aligned to ground truth; (2) robustness to controlled perturbations in geometry, style, and text; and (3) higher-level comprehension, such as recovering a deck's narrative order from shuffled slides. Using publicly available decks from Zenodo (https://huggingface.co/datasets/Forceless/Zenodo10K/viewer/default/pptx), we standardize ground-truth element metadata from PowerPoint XML and live renderings into a unified, verifiable schema. Empirically, VLMs underperform on pixel-accurate extraction and show non-trivial agreement, fidelity, and consistency under controlled perturbations, while performing better on single-slide content understanding; however, they do not reliably capture narrative structure across slides. These results highlight the limits of current VLMs for slide evaluation and motivate calibrated, critic-in-the-loop evaluators that drive iterative refinement and selection in agentic pipelines.

Related papers

• ReDeck: Step-Level Render-Grounded Refinement for Document-to-Slide Generation [50.508786206669605]
  We propose ReDeck, a step-level render-grounded refinement framework for document-to-slide generation.<n>To balance local repair with global quality, ReDeck uses multi-granular feedback for spatial errors, a turn-level adaptive critic for semantic and design guidance, and a submission-level gate for hard layout validation.<n>Across GPT-5.4, Claude-4.6, and Gemini-3.1, ReDeck consistently outperforms existing slide-generation agents.
  arXiv  Detail & Related papers  (2026-08-31T18:12:10Z)
• Slide Deck Q&A Quality Assurance App: A Multi-Stage Pipeline for Pedagogical Question Generation [0.0]
  This paper describes Slide Deck Q&A Quality Assurance (slidesqaqa), a Flask-based software system that extracts text and rendered images from PDF slides.<n>The system reasons jointly about slide modality and pedagogical role, allocates budgets, and revises the deck level to reduce redundancy and improve coverage.<n>Initial experiments indicate that the pipeline can filter non-instructional slides and produce pedagogically coherent questions for visually complex content.
  arXiv  Detail & Related papers  (2026-05-26T01:23:51Z)
• SceneCritic: A Symbolic Evaluator for 3D Indoor Scene Synthesis [6.182838033073788]
  We introduce SceneCritic, a symbolic evaluator for floor-plan-level layouts.<n> SceneCritic's constraints are grounded in SceneOnto, a structured spatial ontology.<n>We show that SceneCritic aligns substantially better with human judgments than VLM-based evaluators.
  arXiv  Detail & Related papers  (2026-04-14T17:59:26Z)
• DeepPresenter: Environment-Grounded Reflection for Agentic Presentation Generation [75.7505732466149]
  DeepPresenter is an agentic framework that adapts to diverse user intents.<n>DeepPresenter autonomously plans, renders, and revises intermediate slide artifacts.
  arXiv  Detail & Related papers  (2026-02-26T10:26:48Z)
• SlidesGen-Bench: Evaluating Slides Generation via Computational and Quantitative Metrics [49.092634600260965]
  We introduce SlidesGen-Bench, a benchmark designed to evaluate slide generation through a lens of three core principles.<n>First, we ground our analysis in the visual domain, treating terminal outputs as renderings to remain agnostic to the underlying generation method.<n>Second, we propose a computational approach that quantitatively assesses slides across three distinct dimensions - Content, Aesthetics, and Editability.
  arXiv  Detail & Related papers  (2026-01-14T13:50:30Z)
• PPTArena: A Benchmark for Agentic PowerPoint Editing [57.55409966131844]
  We introduce a benchmark for PowerPoint editing that measures reliable modifications to real slides under natural-language instructions.<n>PPTArena focuses on in-place editing across 100 decks, 2125 slides, and over 800 targeted edits covering text, charts, tables, animations, and master-level styles.<n>We propose a structure-aware slide-editing agent that plans semantic edit sequences, routes between high-level programmatic tools, and deterministic XML operations for precise control.
  arXiv  Detail & Related papers  (2025-12-02T18:59:50Z)
• PPTBench: Towards Holistic Evaluation of Large Language Models for PowerPoint Layout and Design Understanding [14.793139089425836]
  PowerPoint presentations combine rich textual content with structured visual layouts, making them a natural testbed for evaluating the multimodal reasoning and layout understanding abilities of modern MLLMs.<n>Existing benchmarks focus solely on narrow subtasks while overlooking layout-centric challenges, which are central to real-world slide creation and editing.<n>We introduce PPTBench, a comprehensive multimodal benchmark for evaluating LLMs on PowerPoint-related tasks.
  arXiv  Detail & Related papers  (2025-12-02T10:33:31Z)
• Through the Perspective of LiDAR: A Feature-Enriched and Uncertainty-Aware Annotation Pipeline for Terrestrial Point Cloud Segmentation [5.173182375745059]
  Accurate semantic segmentation of terrestrial laser scanning (TLS) point clouds is limited by costly manual annotation.<n>We propose a semi-automated, uncertainty-aware pipeline that integrates spherical projection, feature enrichment, ensemble learning, and targeted annotation.<n>Our approach projects 3D points to a 2D spherical grid, enriches pixels with multi-source features, and trains an ensemble of segmentation networks to produce pseudo-labels and uncertainty maps.
  arXiv  Detail & Related papers  (2025-10-08T02:25:59Z)
• On the Reliability of Vision-Language Models Under Adversarial Frequency-Domain Perturbations [53.611451075703314]
  Vision-Language Models (VLMs) are increasingly used as perceptual modules for visual content reasoning.<n>We show how these feature transformations undermine authenticity/DeepFake detection and automated image captioning tasks.
  arXiv  Detail & Related papers  (2025-07-30T05:41:29Z)
• Talk to Your Slides: Language-Driven Agents for Efficient Slide Editing [28.792459459465515]
  We propose Talk-to-Your-Slides, an agent to edit slides %in active PowerPoint sessions.<n>Our system enables 34.02% faster processing, 34.76% better instruction fidelity, and 87.42% cheaper operation than baselines.
  arXiv  Detail & Related papers  (2025-05-16T18:12:26Z)
• Presentations are not always linear! GNN meets LLM for Document-to-Presentation Transformation with Attribution [21.473482276335194]
  It is difficult to incorporate such non-linear mapping of content to slides and ensure that the content is faithful to the document. We propose a novel graph based solution where we learn a graph from the input document and use a combination of graph neural network and LLM to generate a presentation.
  arXiv  Detail & Related papers  (2024-05-21T13:52:33Z)
• Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidance [51.30560006045442]
  Image-gRounded guIdaNcE (MARINE) is a framework that is both training-free and API-free.<n>MARINE effectively and efficiently reduces object hallucinations during inference by introducing image-grounded guidance to LVLMs.<n>Our framework's flexibility further allows for the integration of multiple vision models, enabling more reliable and robust object-level guidance.
  arXiv  Detail & Related papers  (2024-02-13T18:59:05Z)
• PIN: Positional Insert Unlocks Object Localisation Abilities in VLMs [55.8550939439138]
  Vision-Language Models (VLMs) have shown immense potential by integrating large language models with vision systems. These models face challenges in the fundamental computer vision task of object localisation, due to their training on multimodal data containing mostly captions. We introduce an input-agnostic Positional Insert (PIN), a learnable spatial prompt, containing a minimal set of parameters that are slid inside the frozen VLM. Our PIN module is trained with a simple next-token prediction task on synthetic data without requiring the introduction of new output heads.
  arXiv  Detail & Related papers  (2024-02-13T18:39:18Z)
• Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models [73.40350756742231]
  Visually-conditioned language models (VLMs) have seen growing adoption in applications such as visual dialogue, scene understanding, and robotic task planning. Despite the volume of new releases, key design decisions around image preprocessing, architecture, and optimization are under-explored.
  arXiv  Detail & Related papers  (2024-02-12T18:21:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.