A Good Talk Does not Look Like a Summary, It Teaches You! Measuring Takeaways from Paper-to-Video Talks

• URL: http://arxiv.org/abs/2606.28531v1
• Date: Fri, 26 Jun 2026 18:30:00 GMT
• Title: A Good Talk Does not Look Like a Summary, It Teaches You! Measuring Takeaways from Paper-to-Video Talks
• Abstract summary: We introduce EffectivePresentationScorer, a framework for evaluating the instructional quality of scientific presentation videos.<n>It checks whether a video explains the main ideas clearly, introduces needed background concepts, and connects technical details to the main contribution of the paper.
• Score: 13.084031365041552
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automatically generated videos from scientific papers are increasingly used for education and research dissemination. However, existing evaluation metrics mainly measure visual quality or whether key points from the paper appear in the video without assessing whether the video actually helps viewers understand the ideas. We introduce EffectivePresentationScorer, a framework for evaluating the instructional quality of scientific presentation videos. It checks whether a video explains the main ideas clearly, introduces needed background concepts, and connects technical details to the main contribution of the paper. When we apply EffectivePresentationScorer to the existing paper-to-video generation systems, we find that generated videos mention the correct topics and follow the structure of the paper but fail to explain prerequisite concepts or clarify why the method works. These failures are often ignored by existing video evaluation metrics, which focus on content presence rather than explanatory quality.

Related papers

• Paper2Video: Automatic Video Generation from Scientific Papers [62.634562246594555]
  Paper2Video is the first benchmark of 101 research papers paired with author-created presentation videos, slides, and speaker metadata.<n>We propose PaperTalker, the first multi-agent framework for academic presentation video generation.
  arXiv  Detail & Related papers  (2025-10-06T17:58:02Z)
• Visual Content Detection in Educational Videos with Transfer Learning and Dataset Enrichment [0.0]
  This paper reports on a transfer learning approach for detecting visual elements in lecture video frames.<n>YOLO was optimized for lecture video object detection with training on multiple benchmark datasets and deploying a semi-supervised auto labeling strategy.
  arXiv  Detail & Related papers  (2025-06-27T04:43:05Z)
• VideoMathQA: Benchmarking Mathematical Reasoning via Multimodal Understanding in Videos [89.39873803375498]
  VideoMathQA is a benchmark designed to evaluate whether models can perform temporally extended cross-modal reasoning on videos.<n>The benchmark spans 10 diverse mathematical domains, covering videos ranging from 10 seconds to over 1 hour.<n>It requires models to interpret structured visual content, understand instructional narratives, and jointly ground concepts across visual, audio, and textual modalities.
  arXiv  Detail & Related papers  (2025-06-05T17:59:58Z)
• Towards A Better Metric for Text-to-Video Generation [102.16250512265995]
  Generative models have demonstrated remarkable capability in synthesizing high-quality text, images, and videos. We introduce a novel evaluation pipeline, the Text-to-Video Score (T2VScore) This metric integrates two pivotal criteria: (1) Text-Video Alignment, which scrutinizes the fidelity of the video in representing the given text description, and (2) Video Quality, which evaluates the video's overall production caliber with a mixture of experts.
  arXiv  Detail & Related papers  (2024-01-15T15:42:39Z)
• Video Summarization Overview [25.465707307283434]
  Video summarization facilitates quickly grasping video content by creating a compact summary of videos. This survey covers early studies as well as recent approaches which take advantage of deep learning techniques.
  arXiv  Detail & Related papers  (2022-10-21T03:29:31Z)
• TL;DW? Summarizing Instructional Videos with Task Relevance & Cross-Modal Saliency [133.75876535332003]
  We focus on summarizing instructional videos, an under-explored area of video summarization. Existing video summarization datasets rely on manual frame-level annotations. We propose an instructional video summarization network that combines a context-aware temporal video encoder and a segment scoring transformer.
  arXiv  Detail & Related papers  (2022-08-14T04:07:40Z)
• Discourse Analysis for Evaluating Coherence in Video Paragraph Captions [99.37090317971312]
  We are exploring a novel discourse based framework to evaluate the coherence of video paragraphs. Central to our approach is the discourse representation of videos, which helps in modeling coherence of paragraphs conditioned on coherence of videos. Our experiment results have shown that the proposed framework evaluates coherence of video paragraphs significantly better than all the baseline methods.
  arXiv  Detail & Related papers  (2022-01-17T04:23:08Z)
• Can We Automate Scientific Reviewing? [89.50052670307434]
  We discuss the possibility of using state-of-the-art natural language processing (NLP) models to generate first-pass peer reviews for scientific papers. We collect a dataset of papers in the machine learning domain, annotate them with different aspects of content covered in each review, and train targeted summarization models that take in papers to generate reviews. Comprehensive experimental results show that system-generated reviews tend to touch upon more aspects of the paper than human-written reviews.
  arXiv  Detail & Related papers  (2021-01-30T07:16:53Z)
• A Clustering-Based Method for Automatic Educational Video Recommendation Using Deep Face-Features of Lecturers [0.0]
  This paper presents a method for generating educational video recommendation using deep face-features of lecturers without identifying them. We use an unsupervised face clustering mechanism to create relations among the videos based on the lecturer's presence. We rank these recommended videos based on the amount of time the referenced lecturers were present.
  arXiv  Detail & Related papers  (2020-10-09T16:53:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.