Listening Deepfake Detection: A New Perspective Beyond Speaking-Centric Forgery Analysis

• URL: http://arxiv.org/abs/2604.12650v1
• Date: Tue, 14 Apr 2026 12:20:07 GMT
• Title: Listening Deepfake Detection: A New Perspective Beyond Speaking-Centric Forgery Analysis
• Abstract summary: We present the task of Listening Deepfake Detection (LDD)<n>We propose MANet, a Motion-aware and Audio-guided Network that captures subtle motion inconsistencies in listener videos.<n>Our work highlights the necessity of rethinking deepfake detection beyond the traditional speaking-centric paradigm.
• Score: 12.060274776566223
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing deepfake detection research has primarily focused on scenarios where the manipulated subject is actively speaking, i.e., generating fabricated content by altering the speaker's appearance or voice. However, in realistic interaction settings, attackers often alternate between falsifying speaking and listening states to mislead their targets, thereby enhancing the realism and persuasiveness of the scenario. Although the detection of 'listening deepfakes' remains largely unexplored and is hindered by a scarcity of both datasets and methodologies, the relatively limited quality of synthesized listening reactions presents an excellent breakthrough opportunity for current deepfake detection efforts. In this paper, we present the task of Listening Deepfake Detection (LDD). We introduce ListenForge, the first dataset specifically designed for this task, constructed using five Listening Head Generation (LHG) methods. To address the distinctive characteristics of listening forgeries, we propose MANet, a Motion-aware and Audio-guided Network that captures subtle motion inconsistencies in listener videos while leveraging speaker's audio semantics to guide cross-modal fusion. Extensive experiments demonstrate that existing Speaking Deepfake Detection (SDD) models perform poorly in listening scenarios. In contrast, MANet achieves significantly superior performance on ListenForge. Our work highlights the necessity of rethinking deepfake detection beyond the traditional speaking-centric paradigm and opens new directions for multimodal forgery analysis in interactive communication settings. The dataset and code are available at https://anonymous.4open.science/r/LDD-B4CB.

Related papers

• From Talking to Singing: A New Challenge for Audio-Visual Deepfake Detection [23.76387977850747]
  Existing methods for audio-visual deepfake detection rely on cross-modal inconsistencies.<n>In singing, rhythmic vocalization weakens this coupling and introduces a nontrivial domain shift.<n>We construct the Singing Head DeepFake dataset using rhythm-aware generative models to fill the gap in singing benchmarks.
  arXiv  Detail & Related papers  (2026-05-27T04:33:39Z)
• Leave No Stone Unturned: Uncovering Holistic Audio-Visual Intrinsic Coherence for Deepfake Detection [27.756028843178708]
  generative AI has enabled hyper-realistic audio-visual deepfakes, intensifying threats to personal security and social trust.<n>Most existing deepfake detectors rely on uni-modal artifacts or audio-visual discrepancies, failing to jointly leverage both sources of information.<n>We argue that robust and generalizable detection should be grounded in intrinsic audio-visual coherence within and across modalities.<n>We introduce HiFi-AVDF, a high-fidelity audio-visual deepfake dataset featuring both text-to-video and image-to-video forgeries from state-of-the-art commercial generators.
  arXiv  Detail & Related papers  (2026-03-25T05:44:25Z)
• Multi-Speaker Conversational Audio Deepfake: Taxonomy, Dataset and Pilot Study [6.567506441691872]
  We introduce a new Multi-speaker Conversational Audio Deepfakes dataset (MsCADD) of 2,830 audio clips containing real and fully synthetic two-speaker conversations.<n>We benchmark three neural baseline models; LFCC-LCNN, RawNet2, and Wav2Vec 2.0 on this dataset and report performance in terms of F1 score, accuracy, true positive rate (TPR), and true negative rate (TNR)<n>Results show that these baseline models provided a useful benchmark, however, the results also highlight that there is a significant gap in multi-speaker deepfake research in reliably detecting synthetic voices
  arXiv  Detail & Related papers  (2026-01-30T20:38:10Z)
• Layover or Direct Flight: Rethinking Audio-Guided Image Segmentation [65.7990140284317]
  We focus on object grounding, i.e., localizing an object of interest in a visual scene based on verbal human instructions.<n>To explore this possibility, we simplify the task by focusing on grounding from single-word spoken instructions.<n>Our results demonstrate that direct grounding from audio is not only feasible but, in some cases, even outperforms transcription-based methods.
  arXiv  Detail & Related papers  (2025-11-27T02:00:28Z)
• SafeEar: Content Privacy-Preserving Audio Deepfake Detection [17.859275594843965]
  We propose SafeEar, a novel framework that aims to detect deepfake audios without relying on accessing the speech content within. Our key idea is to devise a neural audio into a novel decoupling model that well separates the semantic and acoustic information from audio samples. In this way, no semantic content will be exposed to the detector.
  arXiv  Detail & Related papers  (2024-09-14T02:45:09Z)
• Contextual Cross-Modal Attention for Audio-Visual Deepfake Detection and Localization [3.9440964696313485]
  In the digital age, the emergence of deepfakes and synthetic media presents a significant threat to societal and political integrity. Deepfakes based on multi-modal manipulation, such as audio-visual, are more realistic and pose a greater threat. We propose a novel multi-modal attention framework based on recurrent neural networks (RNNs) that leverages contextual information for audio-visual deepfake detection.
  arXiv  Detail & Related papers  (2024-08-02T18:45:01Z)
• What to Remember: Self-Adaptive Continual Learning for Audio Deepfake Detection [53.063161380423715]
  Existing detection models have shown remarkable success in discriminating known deepfake audio, but struggle when encountering new attack types. We propose a continual learning approach called Radian Weight Modification (RWM) for audio deepfake detection.
  arXiv  Detail & Related papers  (2023-12-15T09:52:17Z)
• NPVForensics: Jointing Non-critical Phonemes and Visemes for Deepfake Detection [50.33525966541906]
  Existing multimodal detection methods capture audio-visual inconsistencies to expose Deepfake videos. We propose a novel Deepfake detection method to mine the correlation between Non-critical Phonemes and Visemes, termed NPVForensics. Our model can be easily adapted to the downstream Deepfake datasets with fine-tuning.
  arXiv  Detail & Related papers  (2023-06-12T06:06:05Z)
• Learning to Listen: Modeling Non-Deterministic Dyadic Facial Motion [89.01668641930206]
  We present a framework for modeling interactional communication in dyadic conversations. We autoregressively output multiple possibilities of corresponding listener motion. Our method organically captures the multimodal and non-deterministic nature of nonverbal dyadic interactions.
  arXiv  Detail & Related papers  (2022-04-18T17:58:04Z)
• Voice-Face Homogeneity Tells Deepfake [56.334968246631725]
  Existing detection approaches contribute to exploring the specific artifacts in deepfake videos. We propose to perform the deepfake detection from an unexplored voice-face matching view. Our model obtains significantly improved performance as compared to other state-of-the-art competitors.
  arXiv  Detail & Related papers  (2022-03-04T09:08:50Z)
• Multimodal Attention Fusion for Target Speaker Extraction [108.73502348754842]
  We propose a novel attention mechanism for multi-modal fusion and its training methods. Our proposals improve signal to distortion ratio (SDR) by 1.0 dB over conventional fusion mechanisms on simulated data.
  arXiv  Detail & Related papers  (2021-02-02T05:59:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.