X-AVDT: Audio-Visual Cross-Attention for Robust Deepfake Detection

• URL: http://arxiv.org/abs/2603.08483v1
• Date: Mon, 09 Mar 2026 15:18:42 GMT
• Title: X-AVDT: Audio-Visual Cross-Attention for Robust Deepfake Detection
• Abstract summary: Internal cross-attention mechanisms in generative systems encode fine-grained speech-motion alignment.<n>X-AVDT probes generator-internal audio-visual signals accessed via DDIM inversion to expose these cues.<n>X-AVDT achieves leading performance on MMDF and generalizes strongly to external benchmarks and unseen generators.
• Score: 17.827028185540467
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The surge of highly realistic synthetic videos produced by contemporary generative systems has significantly increased the risk of malicious use, challenging both humans and existing detectors. Against this backdrop, we take a generator-side view and observe that internal cross-attention mechanisms in these models encode fine-grained speech-motion alignment, offering useful correspondence cues for forgery detection. Building on this insight, we propose X-AVDT, a robust and generalizable deepfake detector that probes generator-internal audio-visual signals accessed via DDIM inversion to expose these cues. X-AVDT extracts two complementary signals: (i) a video composite capturing inversion-induced discrepancies, and (ii) an audio-visual cross-attention feature reflecting modality alignment enforced during generation. To enable faithful cross-generator evaluation, we further introduce MMDF, a new multimodal deepfake dataset spanning diverse manipulation types and rapidly evolving synthesis paradigms, including GANs, diffusion, and flow-matching. Extensive experiments demonstrate that X-AVDT achieves leading performance on MMDF and generalizes strongly to external benchmarks and unseen generators, outperforming existing methods with accuracy improved by 13.1%. Our findings highlight the importance of leveraging internal audio-visual consistency cues for robustness to future generators in deepfake detection.

Related papers

• SAVe: Self-Supervised Audio-visual Deepfake Detection Exploiting Visual Artifacts and Audio-visual Misalignment [74.05278327933006]
  We propose SAVe, a self-supervised audio-visual deepfake detection framework that learns entirely on authentic videos.<n>SAVE generates on-the-fly, identity-preserving, region-aware self-blended pseudo-manipulations to emulate tampering artifacts.<n>To capture cross-modal evidence, SAVe also models lip-speech synchronization.
  arXiv  Detail & Related papers  (2026-03-26T08:01:35Z)
• TLDiffGAN: A Latent Diffusion-GAN Framework with Temporal Information Fusion for Anomalous Sound Detection [39.234515088121086]
  We propose a novel framework, TLDiffGAN, which consists of two complementary branches.<n>One branch incorporates a latent diffusion model into the GAN generator for adversarial training, thereby making the discriminator's task more challenging and improving the quality of generated samples.<n>We also introduce a TMixup spectrogram augmentation technique to enhance sensitivity to subtle and localized temporal patterns that are often overlooked.
  arXiv  Detail & Related papers  (2026-02-01T07:04:30Z)
• AdaptPrompt: Parameter-Efficient Adaptation of VLMs for Generalizable Deepfake Detection [7.76090543025328]
  Recent advances in image generation have led to the widespread availability of highly realistic synthetic media, increasing the difficulty of reliable deepfake detection.<n>A key challenge is generalization, as detectors trained on a narrow class of generators often fail when confronted with unseen models.<n>We address the pressing need for generalizable detection by leveraging large vision-language models, specifically CLIP, to identify synthetic content across diverse generative techniques.
  arXiv  Detail & Related papers  (2025-12-19T16:06:03Z)
• Rethinking Cross-Generator Image Forgery Detection through DINOv3 [62.80415066351157]
  Cross-generator detection has emerged as a new challenge forgenerative models.<n>We show that frozen visual foundation models, especially DINOv3, already exhibit strong cross-generator detection capability.<n>We introduce a training-free token-ranking strategy followed by a lightweight linear probe to select a small subset of authenticity-relevant tokens.
  arXiv  Detail & Related papers  (2025-11-27T14:01:50Z)
• Improving Deepfake Detection with Reinforcement Learning-Based Adaptive Data Augmentation [60.04281435591454]
  CRDA (Curriculum Reinforcement-Learning Data Augmentation) is a novel framework guiding detectors to progressively master multi-domain forgery features.<n>Central to our approach is integrating reinforcement learning and causal inference.<n>Our method significantly improves detector generalizability, outperforming SOTA methods across multiple cross-domain datasets.
  arXiv  Detail & Related papers  (2025-11-10T12:45:52Z)
• Breaking Latent Prior Bias in Detectors for Generalizable AIGC Image Detection [11.907536189598577]
  Current AIGC detectors often achieve near-perfect accuracy on images produced by the same generator used for training but struggle to generalize to outputs from unseen generators.<n>We trace this failure in part to latent prior bias: detectors learn shortcuts tied to patterns stemming from the initial noise vector rather than learning robust generative artifacts.<n>We propose On-Manifold Adversarial Training (OMAT), which generates adversarial examples that remain on the generator's output manifold.
  arXiv  Detail & Related papers  (2025-06-01T07:20:45Z)
• D$^3$: Scaling Up Deepfake Detection by Learning from Discrepancy [29.919663502808575]
  Existing literature emphasizes the generalization capability of deepfake detection on unseen generators.<n>This work seeks a step toward a universal deepfake detection system with better generalization and robustness.
  arXiv  Detail & Related papers  (2024-04-06T10:45:02Z)
• GenFace: A Large-Scale Fine-Grained Face Forgery Benchmark and Cross Appearance-Edge Learning [50.7702397913573]
  The rapid advancement of photorealistic generators has reached a critical juncture where the discrepancy between authentic and manipulated images is increasingly indistinguishable. Although there have been a number of publicly available face forgery datasets, the forgery faces are mostly generated using GAN-based synthesis technology. We propose a large-scale, diverse, and fine-grained high-fidelity dataset, namely GenFace, to facilitate the advancement of deepfake detection.
  arXiv  Detail & Related papers  (2024-02-03T03:13:50Z)
• MIS-AVoiDD: Modality Invariant and Specific Representation for Audio-Visual Deepfake Detection [4.659427498118277]
  A novel kind of deepfakes has emerged with either audio or visual modalities manipulated. Existing multimodal deepfake detectors are often based on the fusion of the audio and visual streams from the video. In this paper, we tackle the problem at the representation level to aid the fusion of audio and visual streams for multimodal deepfake detection.
  arXiv  Detail & Related papers  (2023-10-03T17:43:24Z)
• Voice-Face Homogeneity Tells Deepfake [56.334968246631725]
  Existing detection approaches contribute to exploring the specific artifacts in deepfake videos. We propose to perform the deepfake detection from an unexplored voice-face matching view. Our model obtains significantly improved performance as compared to other state-of-the-art competitors.
  arXiv  Detail & Related papers  (2022-03-04T09:08:50Z)
• Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis [69.09526348527203]
  Deep generative models have led to highly realistic media, known as deepfakes, that are commonly indistinguishable from real to human eyes. We propose a novel fake detection that is designed to re-synthesize testing images and extract visual cues for detection. We demonstrate the improved effectiveness, cross-GAN generalization, and robustness against perturbations of our approach in a variety of detection scenarios.
  arXiv  Detail & Related papers  (2021-05-29T21:22:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.