Undercover Deepfakes: Detecting Fake Segments in Videos

• URL: http://arxiv.org/abs/2305.06564v4
• Date: Fri, 25 Aug 2023 03:12:20 GMT
• Title: Undercover Deepfakes: Detecting Fake Segments in Videos
• Authors: Sanjay Saha, Rashindrie Perera, Sachith Seneviratne, Tamasha Malepathirana, Sanka Rasnayaka, Deshani Geethika, Terence Sim, Saman Halgamuge
• Abstract summary: deepfake generation is a new paradigm of deepfakes which are mostly real videos altered slightly to distort the truth. In this paper, we present a deepfake detection method that can address this issue by performing deepfake prediction at the frame and video levels. In particular, the paradigm we address will form a powerful tool for the moderation of deepfakes, where human oversight can be better targeted to the parts of videos suspected of being deepfakes.
• Score: 1.2609216345578933
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The recent renaissance in generative models, driven primarily by the advent of diffusion models and iterative improvement in GAN methods, has enabled many creative applications. However, each advancement is also accompanied by a rise in the potential for misuse. In the arena of the deepfake generation, this is a key societal issue. In particular, the ability to modify segments of videos using such generative techniques creates a new paradigm of deepfakes which are mostly real videos altered slightly to distort the truth. This paradigm has been under-explored by the current deepfake detection methods in the academic literature. In this paper, we present a deepfake detection method that can address this issue by performing deepfake prediction at the frame and video levels. To facilitate testing our method, we prepared a new benchmark dataset where videos have both real and fake frame sequences with very subtle transitions. We provide a benchmark on the proposed dataset with our detection method which utilizes the Vision Transformer based on Scaling and Shifting to learn spatial features, and a Timeseries Transformer to learn temporal features of the videos to help facilitate the interpretation of possible deepfakes. Extensive experiments on a variety of deepfake generation methods show excellent results by the proposed method on temporal segmentation and classical video-level predictions as well. In particular, the paradigm we address will form a powerful tool for the moderation of deepfakes, where human oversight can be better targeted to the parts of videos suspected of being deepfakes. All experiments can be reproduced at: github.com/rgb91/temporal-deepfake-segmentation.

Related papers

• Deepfake detection in videos with multiple faces using geometric-fakeness features [79.16635054977068]
  Deepfakes of victims or public figures can be used by fraudsters for blackmailing, extorsion and financial fraud. In our research we propose to use geometric-fakeness features (GFF) that characterize a dynamic degree of a face presence in a video. We employ our approach to analyze videos with multiple faces that are simultaneously present in a video.
  arXiv  Detail & Related papers  (2024-10-10T13:10:34Z)
• FakeOut: Leveraging Out-of-domain Self-supervision for Multi-modal Video Deepfake Detection [10.36919027402249]
  Synthetic videos of speaking humans can be used to spread misinformation in a convincing manner. FakeOut is a novel approach that relies on multi-modal data throughout both the pre-training phase and the adaption phase. Our method achieves state-of-the-art results in cross-dataset generalization on audio-visual datasets.
  arXiv  Detail & Related papers  (2022-12-01T18:56:31Z)
• DeePhy: On Deepfake Phylogeny [58.01631614114075]
  DeePhy is a novel Deepfake Phylogeny dataset which consists of 5040 deepfake videos generated using three different generation techniques. We present the benchmark on DeePhy dataset using six deepfake detection algorithms.
  arXiv  Detail & Related papers  (2022-09-19T15:30:33Z)
• Deep Convolutional Pooling Transformer for Deepfake Detection [54.10864860009834]
  We propose a deep convolutional Transformer to incorporate decisive image features both locally and globally. Specifically, we apply convolutional pooling and re-attention to enrich the extracted features and enhance efficacy. The proposed solution consistently outperforms several state-of-the-art baselines on both within- and cross-dataset experiments.
  arXiv  Detail & Related papers  (2022-09-12T15:05:41Z)
• Detecting Deepfake by Creating Spatio-Temporal Regularity Disruption [94.5031244215761]
  We propose to boost the generalization of deepfake detection by distinguishing the "regularity disruption" that does not appear in real videos. Specifically, by carefully examining the spatial and temporal properties, we propose to disrupt a real video through a Pseudo-fake Generator. Such practice allows us to achieve deepfake detection without using fake videos and improves the generalization ability in a simple and efficient manner.
  arXiv  Detail & Related papers  (2022-07-21T10:42:34Z)
• Audio-Visual Person-of-Interest DeepFake Detection [77.04789677645682]
  The aim of this work is to propose a deepfake detector that can cope with the wide variety of manipulation methods and scenarios encountered in the real world. We leverage a contrastive learning paradigm to learn the moving-face and audio segment embeddings that are most discriminative for each identity. Our method can detect both single-modality (audio-only, video-only) and multi-modality (audio-video) attacks, and is robust to low-quality or corrupted videos.
  arXiv  Detail & Related papers  (2022-04-06T20:51:40Z)
• Voice-Face Homogeneity Tells Deepfake [56.334968246631725]
  Existing detection approaches contribute to exploring the specific artifacts in deepfake videos. We propose to perform the deepfake detection from an unexplored voice-face matching view. Our model obtains significantly improved performance as compared to other state-of-the-art competitors.
  arXiv  Detail & Related papers  (2022-03-04T09:08:50Z)
• Detecting Deepfake Videos Using Euler Video Magnification [1.8506048493564673]
  Deepfake videos are manipulating videos using advanced machine learning techniques. In this paper, we examine a technique for possible identification of deepfake videos. Our approach uses features extracted from the Euler technique to train three models to classify counterfeit and unaltered videos.
  arXiv  Detail & Related papers  (2021-01-27T17:37:23Z)
• A Convolutional LSTM based Residual Network for Deepfake Video Detection [23.275080108063406]
  We develop a Convolutional LSTM based Residual Network (CLRNet) to detect deepfake videos. We also propose a transfer learning-based approach to generalize different deepfake methods.
  arXiv  Detail & Related papers  (2020-09-16T05:57:06Z)
• Deepfake Detection using Spatiotemporal Convolutional Networks [0.0]
  Deepfake detection methods only use individual frames and therefore fail to learn from temporal information. We created a benchmark of performance using Celeb-DF dataset. Our methods outperformed state-of-theart frame-based detection methods.
  arXiv  Detail & Related papers  (2020-06-26T01:32:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.