Attend to what I say: Highlighting relevant content on slides

• URL: http://arxiv.org/abs/2601.10244v1
• Date: Thu, 15 Jan 2026 10:04:59 GMT
• Title: Attend to what I say: Highlighting relevant content on slides
• Abstract summary: We introduce a method that automatically identifies and highlights the most relevant slide regions based on the speaker's narrative.<n>By analyzing spoken content and matching it with textual or graphical elements in the slides, our approach ensures better synchronization between what listeners hear and what they need to attend to.
• Score: 15.282453684375293
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Imagine sitting in a presentation, trying to follow the speaker while simultaneously scanning the slides for relevant information. While the entire slide is visible, identifying the relevant regions can be challenging. As you focus on one part of the slide, the speaker moves on to a new sentence, leaving you scrambling to catch up visually. This constant back-and-forth creates a disconnect between what is being said and the most important visual elements, making it hard to absorb key details, especially in fast-paced or content-heavy presentations such as conference talks. This requires an understanding of slides, including text, graphics, and layout. We introduce a method that automatically identifies and highlights the most relevant slide regions based on the speaker's narrative. By analyzing spoken content and matching it with textual or graphical elements in the slides, our approach ensures better synchronization between what listeners hear and what they need to attend to. We explore different ways of solving this problem and assess their success and failure cases. Analyzing multimedia documents is emerging as a key requirement for seamless understanding of content-rich videos, such as educational videos and conference talks, by reducing cognitive strain and improving comprehension. Code and dataset are available at: https://github.com/meghamariamkm2002/Slide_Highlight

Related papers

• Slide Deck Q&A Quality Assurance App: A Multi-Stage Pipeline for Pedagogical Question Generation [0.0]
  This paper describes Slide Deck Q&A Quality Assurance (slidesqaqa), a Flask-based software system that extracts text and rendered images from PDF slides.<n>The system reasons jointly about slide modality and pedagogical role, allocates budgets, and revises the deck level to reduce redundancy and improve coverage.<n>Initial experiments indicate that the pipeline can filter non-instructional slides and produce pedagogically coherent questions for visually complex content.
  arXiv  Detail & Related papers  (2026-05-26T01:23:51Z)
• BOOM: Beyond Only One Modality KIT's Multimodal Multilingual Lecture Companion [56.41649972542962]
  We present textbfBOOM, a multilingual lecture companion that jointly translates lecture audio and slides to produce synchronized outputs across three modalities.<n>Our experiments demonstrate that slide-aware transcripts also yield cascading benefits for downstream tasks such as summarization and question answering.
  arXiv  Detail & Related papers  (2025-12-02T14:27:26Z)
• Paper2Video: Automatic Video Generation from Scientific Papers [62.634562246594555]
  Paper2Video is the first benchmark of 101 research papers paired with author-created presentation videos, slides, and speaker metadata.<n>We propose PaperTalker, the first multi-agent framework for academic presentation video generation.
  arXiv  Detail & Related papers  (2025-10-06T17:58:02Z)
• Generating Narrated Lecture Videos from Slides with Synchronized Highlights [55.2480439325792]
  We introduce an end-to-end system designed to automate the process of turning static slides into video lectures.<n>This system synthesizes a video lecture featuring AI-generated narration precisely synchronized with dynamic visual highlights.<n>We demonstrate the system's effectiveness through a technical evaluation using a manually annotated slide dataset with 1000 samples.
  arXiv  Detail & Related papers  (2025-05-05T18:51:53Z)
• MLLM as Video Narrator: Mitigating Modality Imbalance in Video Moment Retrieval [53.417646562344906]
  Video Moment Retrieval (VMR) aims to localize a specific temporal segment within an untrimmed long video given a natural language query. Existing methods often suffer from inadequate training annotations, i.e., the sentence typically matches with a fraction of the prominent video content in the foreground with limited wording diversity. This intrinsic modality imbalance leaves a considerable portion of visual information remaining unaligned with text. In this work, we take an MLLM as a video narrator to generate plausible textual descriptions of the video, thereby mitigating the modality imbalance and boosting the temporal localization.
  arXiv  Detail & Related papers  (2024-06-25T18:39:43Z)
• FastPerson: Enhancing Video Learning through Effective Video Summarization that Preserves Linguistic and Visual Contexts [23.6178079869457]
  We propose FastPerson, a video summarization approach that considers both the visual and auditory information in lecture videos. FastPerson creates summary videos by utilizing audio transcriptions along with on-screen images and text. It reduces viewing time by 53% at the same level of comprehension as that when using traditional video playback methods.
  arXiv  Detail & Related papers  (2024-03-26T14:16:56Z)
• Multimodal Lecture Presentations Dataset: Understanding Multimodality in Educational Slides [57.86931911522967]
  We test the capabilities of machine learning models in multimodal understanding of educational content. Our dataset contains aligned slides and spoken language, for 180+ hours of video and 9000+ slides, with 10 lecturers from various subjects. We introduce PolyViLT, a multimodal transformer trained with a multi-instance learning loss that is more effective than current approaches.
  arXiv  Detail & Related papers  (2022-08-17T05:30:18Z)
• "Notic My Speech" -- Blending Speech Patterns With Multimedia [65.91370924641862]
  We propose a view-temporal attention mechanism to model both the view dependence and the visemic importance in speech recognition and understanding. Our proposed method outperformed the existing work by 4.99% in terms of the viseme error rate. We show that there is a strong correlation between our model's understanding of multi-view speech and the human perception.
  arXiv  Detail & Related papers  (2020-06-12T06:51:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.