Pindrop it! Audio and Visual Deepfake Countermeasures for Robust Detection and Fine Grained-Localization

• URL: http://arxiv.org/abs/2508.08141v2
• Date: Sun, 26 Oct 2025 01:03:16 GMT
• Title: Pindrop it! Audio and Visual Deepfake Countermeasures for Robust Detection and Fine Grained-Localization
• Authors: Nicholas Klein, Hemlata Tak, James Fullwood, Krishna Regmi, Leonidas Spinoulas, Ganesh Sivaraman, Tianxiang Chen, Elie Khoury,
• Abstract summary: This paper presents solutions for the problems of deepfake video classification and localization.<n>The methods were submitted to the ACM 1M Deepfakes Detection Challenge.
• Score: 13.437341095443907
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The field of visual and audio generation is burgeoning with new state-of-the-art methods. This rapid proliferation of new techniques underscores the need for robust solutions for detecting synthetic content in videos. In particular, when fine-grained alterations via localized manipulations are performed in visual, audio, or both domains, these subtle modifications add challenges to the detection algorithms. This paper presents solutions for the problems of deepfake video classification and localization. The methods were submitted to the ACM 1M Deepfakes Detection Challenge, achieving the best performance in the temporal localization task and a top four ranking in the classification task for the TestA split of the evaluation dataset.

Related papers

• Detecting Localized Deepfake Manipulations Using Action Unit-Guided Video Representations [4.449835214520726]
  Deepfake techniques are increasingly narrowing the gap between real and synthetic videos, raising serious privacy and security concerns.<n>This work presents the first detection approach explicitly designed to generalize to localized edits in deepfake videos.<n>Our method achieves a $20%$ improvement in accuracy over current state-of-the-art detection methods.
  arXiv  Detail & Related papers  (2025-03-28T03:49:00Z)
• Detecting the Undetectable: Assessing the Efficacy of Current Spoof Detection Methods Against Seamless Speech Edits [82.8859060022651]
  We introduce the Speech INfilling Edit (SINE) dataset, created with Voicebox.<n>Subjective evaluations confirm that speech edited using this novel technique is more challenging to detect than conventional cut-and-paste methods.<n>Despite human difficulty, experimental results demonstrate that self-supervised-based detectors can achieve remarkable performance in detection, localization, and generalization.
  arXiv  Detail & Related papers  (2025-01-07T14:17:47Z)
• Weakly Supervised Video Anomaly Detection and Localization with Spatio-Temporal Prompts [57.01985221057047]
  This paper introduces a novel method that learnstemporal prompt embeddings for weakly supervised video anomaly detection and localization (WSVADL) based on pre-trained vision-language models (VLMs) Our method achieves state-of-theart performance on three public benchmarks for the WSVADL task.
  arXiv  Detail & Related papers  (2024-08-12T03:31:29Z)
• AVTENet: A Human-Cognition-Inspired Audio-Visual Transformer-Based Ensemble Network for Video Deepfake Detection [49.81915942821647]
  This study introduces the audio-visual transformer-based ensemble network (AVTENet) to detect deepfake videos.<n>For evaluation, we use the recently released benchmark multimodal audio-video FakeAVCeleb dataset.<n>For a detailed analysis, we evaluate AVTENet, its variants, and several existing methods on multiple test sets of the FakeAVCeleb dataset.
  arXiv  Detail & Related papers  (2023-10-19T19:01:26Z)
• Integrating Audio-Visual Features for Multimodal Deepfake Detection [33.51027054306748]
  Deepfakes are AI-generated media in which an image or video has been digitally modified. This paper proposes an audio-visual-based method for deepfake detection, which integrates fine-grained deepfake identification with binary classification.
  arXiv  Detail & Related papers  (2023-10-05T18:19:56Z)
• NPVForensics: Jointing Non-critical Phonemes and Visemes for Deepfake Detection [50.33525966541906]
  Existing multimodal detection methods capture audio-visual inconsistencies to expose Deepfake videos. We propose a novel Deepfake detection method to mine the correlation between Non-critical Phonemes and Visemes, termed NPVForensics. Our model can be easily adapted to the downstream Deepfake datasets with fine-tuning.
  arXiv  Detail & Related papers  (2023-06-12T06:06:05Z)
• Adaptive Fake Audio Detection with Low-Rank Model Squeezing [50.7916414913962]
  Traditional approaches, such as finetuning, are computationally intensive and pose a risk of impairing the acquired knowledge of known fake audio types. We introduce the concept of training low-rank adaptation matrices tailored specifically to the newly emerging fake audio types. Our approach offers several advantages, including reduced storage memory requirements and lower equal error rates.
  arXiv  Detail & Related papers  (2023-06-08T06:06:42Z)
• Glitch in the Matrix: A Large Scale Benchmark for Content Driven Audio-Visual Forgery Detection and Localization [20.46053083071752]
  We propose and benchmark a new dataset, Localized Visual DeepFake (LAV-DF) LAV-DF consists of strategic content-driven audio, visual and audio-visual manipulations. The proposed baseline method, Boundary Aware Temporal Forgery Detection (BA-TFD), is a 3D Convolutional Neural Network-based architecture.
  arXiv  Detail & Related papers  (2023-05-03T08:48:45Z)
• Delving into Sequential Patches for Deepfake Detection [64.19468088546743]
  Recent advances in face forgery techniques produce nearly untraceable deepfake videos, which could be leveraged with malicious intentions. Previous studies has identified the importance of local low-level cues and temporal information in pursuit to generalize well across deepfake methods. We propose the Local- & Temporal-aware Transformer-based Deepfake Detection framework, which adopts a local-to-global learning protocol.
  arXiv  Detail & Related papers  (2022-07-06T16:46:30Z)
• Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization [19.490174583625862]
  We introduce a content-driven audio-visual deepfake dataset, termed Localized Audio Visual DeepFake (LAV-DF) Specifically, the content-driven audio-visual manipulations are performed strategically to change the sentiment polarity of the whole video. Our extensive quantitative and qualitative analysis demonstrates the proposed method's strong performance for temporal forgery localization and deepfake detection tasks.
  arXiv  Detail & Related papers  (2022-04-13T08:02:11Z)
• VideoForensicsHQ: Detecting High-quality Manipulated Face Videos [77.60295082172098]
  We show how the performance of forgery detectors depends on the presence of artefacts that the human eye can see. We introduce a new benchmark dataset for face video forgery detection, of unprecedented quality.
  arXiv  Detail & Related papers  (2020-05-20T21:17:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.