VoD: Learning Volume of Differences for Video-Based Deepfake Detection

• URL: http://arxiv.org/abs/2503.07607v1
• Date: Mon, 10 Mar 2025 17:59:38 GMT
• Title: VoD: Learning Volume of Differences for Video-Based Deepfake Detection
• Authors: Ying Xu, Marius Pedersen, Kiran Raja,
• Abstract summary: This paper introduces a novel Deepfake detention framework, Volume of Differences (VoD)<n>VoD is designed to enhance detection accuracy by exploiting temporal and spatial inconsistencies between consecutive video frames.<n>We evaluate our approach with intra-dataset and cross-dataset testing scenarios on various well-known Deepfake datasets.
• Score: 9.407035514709293
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The rapid development of deep learning and generative AI technologies has profoundly transformed the digital contact landscape, creating realistic Deepfake that poses substantial challenges to public trust and digital media integrity. This paper introduces a novel Deepfake detention framework, Volume of Differences (VoD), designed to enhance detection accuracy by exploiting temporal and spatial inconsistencies between consecutive video frames. VoD employs a progressive learning approach that captures differences across multiple axes through the use of consecutive frame differences (CFD) and a network with stepwise expansions. We evaluate our approach with intra-dataset and cross-dataset testing scenarios on various well-known Deepfake datasets. Our findings demonstrate that VoD excels with the data it has been trained on and shows strong adaptability to novel, unseen data. Additionally, comprehensive ablation studies examine various configurations of segment length, sampling steps, and intervals, offering valuable insights for optimizing the framework. The code for our VoD framework is available at https://github.com/xuyingzhongguo/VoD.

Related papers

• Investigating Memorization in Video Diffusion Models [58.70363256771246]
  Diffusion models, widely used for image and video generation, face a significant limitation: the risk of memorizing and reproducing training data during inference. We first formally define the two types of memorization in VDMs (content memorization and motion memorization) in a practical way. We then introduce new metrics specifically designed to separately assess content and motion memorization in VDMs.
  arXiv  Detail & Related papers  (2024-10-29T02:34:06Z)
• Contextual Cross-Modal Attention for Audio-Visual Deepfake Detection and Localization [3.9440964696313485]
  In the digital age, the emergence of deepfakes and synthetic media presents a significant threat to societal and political integrity. Deepfakes based on multi-modal manipulation, such as audio-visual, are more realistic and pose a greater threat. We propose a novel multi-modal attention framework based on recurrent neural networks (RNNs) that leverages contextual information for audio-visual deepfake detection.
  arXiv  Detail & Related papers  (2024-08-02T18:45:01Z)
• Patch Spatio-Temporal Relation Prediction for Video Anomaly Detection [19.643936110623653]
  Video Anomaly Detection (VAD) aims to identify abnormalities within a specific context and timeframe. Recent deep learning-based VAD models have shown promising results by generating high-resolution frames. We propose a self-supervised learning approach for VAD through an inter-patch relationship prediction task.
  arXiv  Detail & Related papers  (2024-03-28T03:07:16Z)
• FutureDepth: Learning to Predict the Future Improves Video Depth Estimation [46.421154770321266]
  FutureDepth is a video depth estimation approach that implicitly leverage multi-frame and motion cues to improve depth estimation. We show that FutureDepth significantly improves upon baseline models, outperforms existing video depth estimation methods, and sets new state-of-the-art (SOTA) accuracy.
  arXiv  Detail & Related papers  (2024-03-19T17:55:22Z)
• AVTENet: Audio-Visual Transformer-based Ensemble Network Exploiting Multiple Experts for Video Deepfake Detection [53.448283629898214]
  The recent proliferation of hyper-realistic deepfake videos has drawn attention to the threat of audio and visual forgeries. Most previous work on detecting AI-generated fake videos only utilize visual modality or audio modality. We propose an Audio-Visual Transformer-based Ensemble Network (AVTENet) framework that considers both acoustic manipulation and visual manipulation.
  arXiv  Detail & Related papers  (2023-10-19T19:01:26Z)
• NVDS+: Towards Efficient and Versatile Neural Stabilizer for Video Depth Estimation [58.21817572577012]
  Video depth estimation aims to infer temporally consistent depth. We introduce NVDS+ that stabilizes inconsistent depth estimated by various single-image models in a plug-and-play manner. We also elaborate a large-scale Video Depth in the Wild dataset, which contains 14,203 videos with over two million frames.
  arXiv  Detail & Related papers  (2023-07-17T17:57:01Z)
• Bilevel Fast Scene Adaptation for Low-Light Image Enhancement [50.639332885989255]
  Enhancing images in low-light scenes is a challenging but widely concerned task in the computer vision. Main obstacle lies in the modeling conundrum from distribution discrepancy across different scenes. We introduce the bilevel paradigm to model the above latent correspondence. A bilevel learning framework is constructed to endow the scene-irrelevant generality of the encoder towards diverse scenes.
  arXiv  Detail & Related papers  (2023-06-02T08:16:21Z)
• Glitch in the Matrix: A Large Scale Benchmark for Content Driven Audio-Visual Forgery Detection and Localization [20.46053083071752]
  We propose and benchmark a new dataset, Localized Visual DeepFake (LAV-DF) LAV-DF consists of strategic content-driven audio, visual and audio-visual manipulations. The proposed baseline method, Boundary Aware Temporal Forgery Detection (BA-TFD), is a 3D Convolutional Neural Network-based architecture.
  arXiv  Detail & Related papers  (2023-05-03T08:48:45Z)
• Deep Convolutional Pooling Transformer for Deepfake Detection [54.10864860009834]
  We propose a deep convolutional Transformer to incorporate decisive image features both locally and globally. Specifically, we apply convolutional pooling and re-attention to enrich the extracted features and enhance efficacy. The proposed solution consistently outperforms several state-of-the-art baselines on both within- and cross-dataset experiments.
  arXiv  Detail & Related papers  (2022-09-12T15:05:41Z)
• Unsupervised Video Domain Adaptation for Action Recognition: A Disentanglement Perspective [37.45565756522847]
  We consider the generation of cross-domain videos from two sets of latent factors. TranSVAE framework is then developed to model such generation. Experiments on the UCF-HMDB, Jester, and Epic-Kitchens datasets verify the effectiveness and superiority of TranSVAE.
  arXiv  Detail & Related papers  (2022-08-15T17:59:31Z)
• Emotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective Cues [75.1731999380562]
  We present a learning-based method for detecting real and fake deepfake multimedia content. We extract and analyze the similarity between the two audio and visual modalities from within the same video. We compare our approach with several SOTA deepfake detection methods and report per-video AUC of 84.4% on the DFDC and 96.6% on the DF-TIMIT datasets.
  arXiv  Detail & Related papers  (2020-03-14T22:07:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.