Beyond Seeing Is Believing: On Crowdsourced Detection of Audiovisual Deepfakes

• URL: http://arxiv.org/abs/2605.04797v1
• Date: Wed, 06 May 2026 11:48:04 GMT
• Title: Beyond Seeing Is Believing: On Crowdsourced Detection of Audiovisual Deepfakes
• Abstract summary: We study audiovisual deepfake detection by measuring how consistently crowd workers distinguish authentic from manipulated videos.<n>We run two matched crowdsourcing studies on Prolific using AV-Deepfake1M and the Trusted Media Challenge dataset.<n>Results show that crowd workers rarely misclassify authentic videos as manipulated, but they miss many manipulations, and agreement remains limited across videos.
• Score: 4.186575888568895
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deepfakes are increasingly realistic and easy to produce, raising concerns about the reliability of human judgments in misinformation settings. We study audiovisual deepfake detection by measuring how consistently crowd workers distinguish authentic from manipulated videos and, when they flag a video as manipulated, how accurately they identify the manipulation type (audio-only, video-only, or audio-video) and how consistently they report manipulation timestamps. We run two matched crowdsourcing studies on Prolific using AV-Deepfake1M and the Trusted Media Challenge (TMC) dataset. We sample 48 videos per dataset (96 total) and collect 960 judgments (10 per video). Results show that crowd workers rarely misclassify authentic videos as manipulated, but they miss many manipulations, and agreement remains limited across videos. Aggregating multiple judgments per video stabilizes the authenticity signal, but it cannot recover manipulations that most workers consistently miss. Manipulation type identification is substantially noisier than authenticity detection even when workers detect a manipulation, with joint audio-video cases being particularly hard to recognize. Overall, these findings suggest that crowdsourcing can provide a scalable screening signal for audiovisual authenticity, while reliable modality attribution remains an open challenge.

Related papers

• When Seeing Is Not Believing -- A Benchmark for Search-Grounded Video Misinformation Detection [21.933691657055075]
  EVID-Bench is a benchmark for search-grounded video misinformation detection.<n>It comprises 222 videos spanning 9 manipulation types across 3 categories: AI generation, single-source editing, and multi-source editing.<n>The best system achieves only 61.43% point-level accuracy and 43.24% video-level accuracy, while AI-generated manipulations remain especially challenging.
  arXiv  Detail & Related papers  (2026-06-02T18:03:35Z)
• SAVe: Self-Supervised Audio-visual Deepfake Detection Exploiting Visual Artifacts and Audio-visual Misalignment [74.05278327933006]
  We propose SAVe, a self-supervised audio-visual deepfake detection framework that learns entirely on authentic videos.<n>SAVE generates on-the-fly, identity-preserving, region-aware self-blended pseudo-manipulations to emulate tampering artifacts.<n>To capture cross-modal evidence, SAVe also models lip-speech synchronization.
  arXiv  Detail & Related papers  (2026-03-26T08:01:35Z)
• Circumventing shortcuts in audio-visual deepfake detection datasets with unsupervised learning [3.453303606167197]
  We show that two of the most widely used audio-video deepfake datasets suffer from a previously unidentified spurious feature: the leading silence.<n>Fake videos start with a very brief moment of silence and based on this feature alone, we can separate the real and fake samples almost perfectly.<n>We propose a shift from supervised to unsupervised learning by training models exclusively on real data.
  arXiv  Detail & Related papers  (2024-11-29T18:58:20Z)
• Deepfake detection in videos with multiple faces using geometric-fakeness features [79.16635054977068]
  Deepfakes of victims or public figures can be used by fraudsters for blackmailing, extorsion and financial fraud. In our research we propose to use geometric-fakeness features (GFF) that characterize a dynamic degree of a face presence in a video. We employ our approach to analyze videos with multiple faces that are simultaneously present in a video.
  arXiv  Detail & Related papers  (2024-10-10T13:10:34Z)
• AVTENet: A Human-Cognition-Inspired Audio-Visual Transformer-Based Ensemble Network for Video Deepfake Detection [49.81915942821647]
  This study introduces the audio-visual transformer-based ensemble network (AVTENet) to detect deepfake videos.<n>For evaluation, we use the recently released benchmark multimodal audio-video FakeAVCeleb dataset.<n>For a detailed analysis, we evaluate AVTENet, its variants, and several existing methods on multiple test sets of the FakeAVCeleb dataset.
  arXiv  Detail & Related papers  (2023-10-19T19:01:26Z)
• Self-Supervised Video Forensics by Audio-Visual Anomaly Detection [19.842795378751923]
  Manipulated videos often contain subtle inconsistencies between their visual and audio signals. We propose a video forensics method, based on anomaly detection, that can identify these inconsistencies. We train an autoregressive model to generate sequences of audio-visual features, using feature sets that capture the temporal synchronization between video frames and sound.
  arXiv  Detail & Related papers  (2023-01-04T18:59:49Z)
• FakeOut: Leveraging Out-of-domain Self-supervision for Multi-modal Video Deepfake Detection [10.36919027402249]
  Synthetic videos of speaking humans can be used to spread misinformation in a convincing manner. FakeOut is a novel approach that relies on multi-modal data throughout both the pre-training phase and the adaption phase. Our method achieves state-of-the-art results in cross-dataset generalization on audio-visual datasets.
  arXiv  Detail & Related papers  (2022-12-01T18:56:31Z)
• SceneFake: An Initial Dataset and Benchmarks for Scene Fake Audio Detection [54.74467470358476]
  This paper proposes a dataset for scene fake audio detection named SceneFake. A manipulated audio is generated by only tampering with the acoustic scene of an original audio. Some scene fake audio detection benchmark results on the SceneFake dataset are reported in this paper.
  arXiv  Detail & Related papers  (2022-11-11T09:05:50Z)
• Audio-Visual Person-of-Interest DeepFake Detection [77.04789677645682]
  The aim of this work is to propose a deepfake detector that can cope with the wide variety of manipulation methods and scenarios encountered in the real world. We leverage a contrastive learning paradigm to learn the moving-face and audio segment embeddings that are most discriminative for each identity. Our method can detect both single-modality (audio-only, video-only) and multi-modality (audio-video) attacks, and is robust to low-quality or corrupted videos.
  arXiv  Detail & Related papers  (2022-04-06T20:51:40Z)
• Voice-Face Homogeneity Tells Deepfake [56.334968246631725]
  Existing detection approaches contribute to exploring the specific artifacts in deepfake videos. We propose to perform the deepfake detection from an unexplored voice-face matching view. Our model obtains significantly improved performance as compared to other state-of-the-art competitors.
  arXiv  Detail & Related papers  (2022-03-04T09:08:50Z)
• Watch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion [82.06128362686445]
  We propose a multi-modal semantic forensic approach to handle both cheapfakes and visually persuasive deepfakes. We leverage the idea of attribution to learn person-specific biometric patterns that distinguish a given speaker from others. Unlike existing person-specific approaches, our method is also effective against attacks that focus on lip manipulation.
  arXiv  Detail & Related papers  (2021-12-21T01:57:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.