Misinformation Span Detection in Videos via Audio Transcripts

• URL: http://arxiv.org/abs/2604.21767v1
• Date: Thu, 23 Apr 2026 15:23:13 GMT
• Title: Misinformation Span Detection in Videos via Audio Transcripts
• Abstract summary: We create two novel datasets that allow us to explore misinformation detection on videos via audio transcripts.<n>We transcribe each video's audio to text, identifying the video segment in which the misinformation claims appears.<n>Our results show that we can identify in which part of a video there is misinformation with an F1 score of 0.68.
• Score: 6.262239090050024
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Online misinformation is one of the most challenging issues lately, yielding severe consequences, including political polarization, attacks on democracy, and public health risks. Misinformation manifests in any platform with a large user base, including online social networks and messaging apps. It permeates all media and content forms, including images, text, audio, and video. Distinctly, video-based misinformation represents a multifaceted challenge for fact-checkers, given the ease with which individuals can record and upload videos on various video-sharing platforms. Previous research efforts investigated detecting video-based misinformation, focusing on whether a video shares misinformation or not on a video level. While this approach is useful, it only provides a limited and non-easily interpretable view of the problem given that it does not provide an additional context of when misinformation occurs within videos and what content (i.e., claims) are responsible for the video's misinformation nature. In this work, we attempt to bridge this research gap by creating two novel datasets that allow us to explore misinformation detection on videos via audio transcripts, focusing on identifying the span of videos that are responsible for the video's misinformation claim (misinformation span detection). We present two new datasets for this task. We transcribe each video's audio to text, identifying the video segment in which the misinformation claims appears, resulting in two datasets of more than 500 videos with over 2,400 segments containing annotated fact-checked claims. Then, we employ classifiers built with state-of-the-art language models, and our results show that we can identify in which part of a video there is misinformation with an F1 score of 0.68. We make publicly available our annotated datasets. We also release all transcripts, audio and videos.

Related papers

• MultiVENT 2.0: A Massive Multilingual Benchmark for Event-Centric Video Retrieval [57.891157692501345]
  $textbfMultiVENT 2.0$ is a large-scale, multilingual event-centric video retrieval benchmark.<n>It features a collection of more than 218,000 news videos and 3,906 queries targeting specific world events.<n>Preliminary results show that state-of-the-art vision-language models struggle significantly with this task.
  arXiv  Detail & Related papers  (2024-10-15T13:56:34Z)
• Multi-modal News Understanding with Professionally Labelled Videos (ReutersViLNews) [25.78619140103048]
  We present a large-scale analysis on an in-house dataset collected by the Reuters News Agency, called Reuters Video-Language News (ReutersViLNews) dataset. The dataset focuses on high-level video-language understanding with an emphasis on long-form news. The results suggest that news-oriented videos are a substantial challenge for current video-language understanding algorithms.
  arXiv  Detail & Related papers  (2024-01-23T00:42:04Z)
• Not all Fake News is Written: A Dataset and Analysis of Misleading Video Headlines [6.939987423356328]
  We present a dataset that consists of videos and whether annotators believe the headline is representative of the video's contents. After collecting and annotating this dataset, we analyze multimodal baselines for detecting misleading headlines. Our annotation process also focuses on why annotators view a video as misleading, allowing us to better understand the interplay of annotators' background and the content of the videos.
  arXiv  Detail & Related papers  (2023-10-20T23:47:01Z)
• InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation [90.71796406228265]
  InternVid is a large-scale video-centric multimodal dataset that enables learning powerful and transferable video-text representations. The InternVid dataset contains over 7 million videos lasting nearly 760K hours, yielding 234M video clips accompanied by detailed descriptions of total 4.1B words.
  arXiv  Detail & Related papers  (2023-07-13T17:58:32Z)
• Combating Online Misinformation Videos: Characterization, Detection, and Future Directions [13.960032991158402]
  Video-based misinformation poses a new threat to the health of the online information ecosystem. We first analyze and characterize the misinformation video from three levels including signals, semantics, and intents. We introduce existing resources including representative datasets and useful tools.
  arXiv  Detail & Related papers  (2023-02-07T04:03:55Z)
• Video-Guided Curriculum Learning for Spoken Video Grounding [65.49979202728167]
  We introduce a new task, spoken video grounding (SVG), which aims to localize the desired video fragments from spoken language descriptions. To rectify the discriminative phonemes and extract video-related information from noisy audio, we develop a novel video-guided curriculum learning (VGCL) In addition, we collect the first large-scale spoken video grounding dataset based on ActivityNet.
  arXiv  Detail & Related papers  (2022-09-01T07:47:01Z)
• Misinformation Detection in Social Media Video Posts [0.4724825031148411]
  Short-form video by social media platforms has become a critical challenge for social media providers. We develop methods to detect misinformation in social media posts, exploiting modalities such as video and text. We collect 160,000 video posts from Twitter, and leverage self-supervised learning to learn expressive representations of joint visual and textual data.
  arXiv  Detail & Related papers  (2022-02-15T20:14:54Z)
• Misinformation Detection on YouTube Using Video Captions [6.503828590815483]
  This work proposes an approach that uses state-of-the-art NLP techniques to extract features from video captions (subtitles) To evaluate our approach, we utilize a publicly accessible and labeled dataset for classifying videos as misinformation or not.
  arXiv  Detail & Related papers  (2021-07-02T10:02:36Z)
• Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions [75.77044856100349]
  We present the Spoken Moments dataset of 500k spoken captions each attributed to a unique short video depicting a broad range of different events. We show that our AMM approach consistently improves our results and that models trained on our Spoken Moments dataset generalize better than those trained on other video-caption datasets.
  arXiv  Detail & Related papers  (2021-05-10T16:30:46Z)
• QuerYD: A video dataset with high-quality text and audio narrations [85.6468286746623]
  We introduce QuerYD, a new large-scale dataset for retrieval and event localisation in video. A unique feature of our dataset is the availability of two audio tracks for each video: the original audio, and a high-quality spoken description. The dataset is based on YouDescribe, a volunteer project that assists visually-impaired people by attaching voiced narrations to existing YouTube videos.
  arXiv  Detail & Related papers  (2020-11-22T17:33:44Z)
• VIOLIN: A Large-Scale Dataset for Video-and-Language Inference [103.7457132841367]
  We introduce a new task, Video-and-Language Inference, for joint multimodal understanding of video and text. Given a video clip with subtitles aligned as premise, paired with a natural language hypothesis based on the video content, a model needs to infer whether the hypothesis is entailed or contradicted by the given video clip. A new large-scale dataset, named Violin (VIdeO-and-Language INference), is introduced for this task, which consists of 95,322 video-hypothesis pairs from 15,887 video clips.
  arXiv  Detail & Related papers  (2020-03-25T20:39:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.