Exploiting Style Latent Flows for Generalizing Deepfake Video Detection

• URL: http://arxiv.org/abs/2403.06592v3
• Date: Mon, 20 May 2024 13:01:23 GMT
• Title: Exploiting Style Latent Flows for Generalizing Deepfake Video Detection
• Authors: Jongwook Choi, Taehoon Kim, Yonghyun Jeong, Seungryul Baek, Jongwon Choi,
• Abstract summary: We present a new approach for the detection of fake videos, based on the analysis of style latent vectors and their abnormal behavior in temporal changes in the generated videos. Our framework utilizes the StyleGRU module, trained by contrastive learning, to represent the dynamic properties of style latent vectors.
• Score: 17.47632743516689
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents a new approach for the detection of fake videos, based on the analysis of style latent vectors and their abnormal behavior in temporal changes in the generated videos. We discovered that the generated facial videos suffer from the temporal distinctiveness in the temporal changes of style latent vectors, which are inevitable during the generation of temporally stable videos with various facial expressions and geometric transformations. Our framework utilizes the StyleGRU module, trained by contrastive learning, to represent the dynamic properties of style latent vectors. Additionally, we introduce a style attention module that integrates StyleGRU-generated features with content-based features, enabling the detection of visual and temporal artifacts. We demonstrate our approach across various benchmark scenarios in deepfake detection, showing its superiority in cross-dataset and cross-manipulation scenarios. Through further analysis, we also validate the importance of using temporal changes of style latent vectors to improve the generality of deepfake video detection.

Related papers

• Deepfake Detection with Spatio-Temporal Consistency and Attention [46.1135899490656]
  Deepfake videos are causing growing concerns among communities due to their ever-increasing realism. Current methods for detecting forged videos rely mainly on global frame features. We propose a neural Deepfake detector that focuses on the localized manipulative signatures of the forged videos.
  arXiv  Detail & Related papers  (2025-02-12T08:51:33Z)
• Extending Information Bottleneck Attribution to Video Sequences [4.996373299748921]
  We introduce VIBA, a novel approach for explainable video classification by adapting Information Bottlenecks for Attribution to video sequences. Our results show that VIBA generates temporally and spatially consistent explanations, which align closely with human annotations.
  arXiv  Detail & Related papers  (2025-01-28T12:19:44Z)
• Weakly Supervised Video Anomaly Detection and Localization with Spatio-Temporal Prompts [57.01985221057047]
  This paper introduces a novel method that learnstemporal prompt embeddings for weakly supervised video anomaly detection and localization (WSVADL) based on pre-trained vision-language models (VLMs) Our method achieves state-of-theart performance on three public benchmarks for the WSVADL task.
  arXiv  Detail & Related papers  (2024-08-12T03:31:29Z)
• Dynamic Erasing Network Based on Multi-Scale Temporal Features for Weakly Supervised Video Anomaly Detection [103.92970668001277]
  We propose a Dynamic Erasing Network (DE-Net) for weakly supervised video anomaly detection. We first propose a multi-scale temporal modeling module, capable of extracting features from segments of varying lengths. Then, we design a dynamic erasing strategy, which dynamically assesses the completeness of the detected anomalies.
  arXiv  Detail & Related papers  (2023-12-04T09:40:11Z)
• Implicit Motion Handling for Video Camouflaged Object Detection [60.98467179649398]
  We propose a new video camouflaged object detection (VCOD) framework. It can exploit both short-term and long-term temporal consistency to detect camouflaged objects from video frames.
  arXiv  Detail & Related papers  (2022-03-14T17:55:41Z)
• PreViTS: Contrastive Pretraining with Video Tracking Supervision [53.73237606312024]
  PreViTS is an unsupervised SSL framework for selecting clips containing the same object. PreViTS spatially constrains the frame regions to learn from and trains the model to locate meaningful objects. We train a momentum contrastive (MoCo) encoder on VGG-Sound and Kinetics-400 datasets with PreViTS.
  arXiv  Detail & Related papers  (2021-12-01T19:49:57Z)
• Video Salient Object Detection via Contrastive Features and Attention Modules [106.33219760012048]
  We propose a network with attention modules to learn contrastive features for video salient object detection. A co-attention formulation is utilized to combine the low-level and high-level features. We show that the proposed method requires less computation, and performs favorably against the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-11-03T17:40:32Z)
• Weakly Supervised Video Salient Object Detection [79.51227350937721]
  We present the first weakly supervised video salient object detection model based on relabeled "fixation guided scribble annotations" An "Appearance-motion fusion module" and bidirectional ConvLSTM based framework are proposed to achieve effective multi-modal learning and long-term temporal context modeling.
  arXiv  Detail & Related papers  (2021-04-06T09:48:38Z)
• Learning Long-Term Style-Preserving Blind Video Temporal Consistency [6.6908747077585105]
  We propose a postprocessing model, to the transformation applied to videos, in the form of a recurrent neural network. Our model is trained using a Ping Pong procedure and its corresponding loss, recently introduced for GAN video generation. We evaluate our model on the DAVIS and videvo.net datasets and show that our approach offers state-of-the-art results concerning flicker removal.
  arXiv  Detail & Related papers  (2021-03-12T13:54:34Z)
• Spatio-temporal Features for Generalized Detection of Deepfake Videos [12.453288832098314]
  We propose-temporal features, modeled by 3D CNNs, to extend the capabilities to detect new sorts of deep videos. We show that our approach outperforms existing methods in terms of generalization capabilities.
  arXiv  Detail & Related papers  (2020-10-22T16:28:50Z)
• Dynamic texture analysis for detecting fake faces in video sequences [6.1356022122903235]
  This work explores the analysis of texture-temporal dynamics of the video signal. The goal is to characterizing and distinguishing real fake sequences. We propose to build multiple binary decision on the joint analysis of temporal segments.
  arXiv  Detail & Related papers  (2020-07-30T07:21:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.