Deepfake detection in videos with multiple faces using geometric-fakeness features

• URL: http://arxiv.org/abs/2410.07888v1
• Date: Thu, 10 Oct 2024 13:10:34 GMT
• Title: Deepfake detection in videos with multiple faces using geometric-fakeness features
• Authors: Kirill Vyshegorodtsev, Dmitry Kudiyarov, Alexander Balashov, Alexander Kuzmin,
• Abstract summary: Deepfakes of victims or public figures can be used by fraudsters for blackmailing, extorsion and financial fraud. In our research we propose to use geometric-fakeness features (GFF) that characterize a dynamic degree of a face presence in a video. We employ our approach to analyze videos with multiple faces that are simultaneously present in a video.
• Score: 79.16635054977068
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Due to the development of facial manipulation techniques in recent years deepfake detection in video stream became an important problem for face biometrics, brand monitoring or online video conferencing solutions. In case of a biometric authentication, if you replace a real datastream with a deepfake, you can bypass a liveness detection system. Using a deepfake in a video conference, you can penetrate into a private meeting. Deepfakes of victims or public figures can also be used by fraudsters for blackmailing, extorsion and financial fraud. Therefore, the task of detecting deepfakes is relevant to ensuring privacy and security. In existing approaches to a deepfake detection their performance deteriorates when multiple faces are present in a video simultaneously or when there are other objects erroneously classified as faces. In our research we propose to use geometric-fakeness features (GFF) that characterize a dynamic degree of a face presence in a video and its per-frame deepfake scores. To analyze temporal inconsistencies in GFFs between the frames we train a complex deep learning model that outputs a final deepfake prediction. We employ our approach to analyze videos with multiple faces that are simultaneously present in a video. Such videos often occur in practice e.g., in an online video conference. In this case, real faces appearing in a frame together with a deepfake face will significantly affect a deepfake detection and our approach allows to counter this problem. Through extensive experiments we demonstrate that our approach outperforms current state-of-the-art methods on popular benchmark datasets such as FaceForensics++, DFDC, Celeb-DF and WildDeepFake. The proposed approach remains accurate when trained to detect multiple different deepfake generation techniques.

Related papers

• Shaking the Fake: Detecting Deepfake Videos in Real Time via Active Probes [3.6308756891251392]
  Real-time deepfake, a type of generative AI, is capable of "creating" non-existing contents (e.g., swapping one's face with another) in a video. It has been misused to produce deepfake videos for malicious purposes, including financial scams and political misinformation. We propose SFake, a new real-time deepfake detection method that exploits deepfake models' inability to adapt to physical interference.
  arXiv  Detail & Related papers  (2024-09-17T04:58:30Z)
• Undercover Deepfakes: Detecting Fake Segments in Videos [1.2609216345578933]
  deepfake generation is a new paradigm of deepfakes which are mostly real videos altered slightly to distort the truth. In this paper, we present a deepfake detection method that can address this issue by performing deepfake prediction at the frame and video levels. In particular, the paradigm we address will form a powerful tool for the moderation of deepfakes, where human oversight can be better targeted to the parts of videos suspected of being deepfakes.
  arXiv  Detail & Related papers  (2023-05-11T04:43:10Z)
• Audio-Visual Person-of-Interest DeepFake Detection [77.04789677645682]
  The aim of this work is to propose a deepfake detector that can cope with the wide variety of manipulation methods and scenarios encountered in the real world. We leverage a contrastive learning paradigm to learn the moving-face and audio segment embeddings that are most discriminative for each identity. Our method can detect both single-modality (audio-only, video-only) and multi-modality (audio-video) attacks, and is robust to low-quality or corrupted videos.
  arXiv  Detail & Related papers  (2022-04-06T20:51:40Z)
• Voice-Face Homogeneity Tells Deepfake [56.334968246631725]
  Existing detection approaches contribute to exploring the specific artifacts in deepfake videos. We propose to perform the deepfake detection from an unexplored voice-face matching view. Our model obtains significantly improved performance as compared to other state-of-the-art competitors.
  arXiv  Detail & Related papers  (2022-03-04T09:08:50Z)
• Watch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion [82.06128362686445]
  We propose a multi-modal semantic forensic approach to handle both cheapfakes and visually persuasive deepfakes. We leverage the idea of attribution to learn person-specific biometric patterns that distinguish a given speaker from others. Unlike existing person-specific approaches, our method is also effective against attacks that focus on lip manipulation.
  arXiv  Detail & Related papers  (2021-12-21T01:57:04Z)
• Understanding the Security of Deepfake Detection [23.118012417901078]
  We study the security of state-of-the-art deepfake detection methods in adversarial settings. We use two large-scale public deepfakes data sources including FaceForensics++ and Facebook Deepfake Detection Challenge. Our results uncover multiple security limitations of the deepfake detection methods in adversarial settings.
  arXiv  Detail & Related papers  (2021-07-05T14:18:21Z)
• WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection [82.42495493102805]
  We introduce a new dataset WildDeepfake which consists of 7,314 face sequences extracted from 707 deepfake videos collected completely from the internet. We conduct a systematic evaluation of a set of baseline detection networks on both existing and our WildDeepfake datasets, and show that WildDeepfake is indeed a more challenging dataset, where the detection performance can decrease drastically.
  arXiv  Detail & Related papers  (2021-01-05T11:10:32Z)
• A Convolutional LSTM based Residual Network for Deepfake Video Detection [23.275080108063406]
  We develop a Convolutional LSTM based Residual Network (CLRNet) to detect deepfake videos. We also propose a transfer learning-based approach to generalize different deepfake methods.
  arXiv  Detail & Related papers  (2020-09-16T05:57:06Z)
• Deepfake detection: humans vs. machines [4.485016243130348]
  We present a subjective study conducted in a crowdsourcing-like scenario, which systematically evaluates how hard it is for humans to see if the video is deepfake or not. For each video, a simple question: "Is face of the person in the video real of fake?" was answered on average by 19 na"ive subjects. The evaluation demonstrates that while the human perception is very different from the perception of a machine, both successfully but in different ways are fooled by deepfakes.
  arXiv  Detail & Related papers  (2020-09-07T15:20:37Z)
• Sharp Multiple Instance Learning for DeepFake Video Detection [54.12548421282696]
  We introduce a new problem of partial face attack in DeepFake video, where only video-level labels are provided but not all the faces in the fake videos are manipulated. A sharp MIL (S-MIL) is proposed which builds direct mapping from instance embeddings to bag prediction. Experiments on FFPMS and widely used DFDC dataset verify that S-MIL is superior to other counterparts for partially attacked DeepFake video detection.
  arXiv  Detail & Related papers  (2020-08-11T08:52:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.