Face Forgery Detection Based on Facial Region Displacement Trajectory Series

• URL: http://arxiv.org/abs/2212.03678v1
• Date: Wed, 7 Dec 2022 14:47:54 GMT
• Title: Face Forgery Detection Based on Facial Region Displacement Trajectory Series
• Authors: YuYang Sun, ZhiYong Zhang, Isao Echizen, Huy H.Nguyen, ChangZhen Qiu and Lu Sun
• Abstract summary: We develop a method for detecting manipulated videos based on the trajectory of the facial region displacement. This information was used to construct a network for exposing multidimensional artifacts in the trajectory sequences of manipulated videos.
• Score: 10.338298543908339
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep-learning-based technologies such as deepfakes ones have been attracting widespread attention in both society and academia, particularly ones used to synthesize forged face images. These automatic and professional-skill-free face manipulation technologies can be used to replace the face in an original image or video with any target object while maintaining the expression and demeanor. Since human faces are closely related to identity characteristics, maliciously disseminated identity manipulated videos could trigger a crisis of public trust in the media and could even have serious political, social, and legal implications. To effectively detect manipulated videos, we focus on the position offset in the face blending process, resulting from the forced affine transformation of the normalized forged face. We introduce a method for detecting manipulated videos that is based on the trajectory of the facial region displacement. Specifically, we develop a virtual-anchor-based method for extracting the facial trajectory, which can robustly represent displacement information. This information was used to construct a network for exposing multidimensional artifacts in the trajectory sequences of manipulated videos that is based on dual-stream spatial-temporal graph attention and a gated recurrent unit backbone. Testing of our method on various manipulation datasets demonstrated that its accuracy and generalization ability is competitive with that of the leading detection methods.

Related papers

• Learning Expressive And Generalizable Motion Features For Face Forgery Detection [52.54404879581527]
  We propose an effective sequence-based forgery detection framework based on an existing video classification method. To make the motion features more expressive for manipulation detection, we propose an alternative motion consistency block. We make a general video classification network achieve promising results on three popular face forgery datasets.
  arXiv  Detail & Related papers  (2024-03-08T09:25:48Z)
• Mover: Mask and Recovery based Facial Part Consistency Aware Method for Deepfake Video Detection [33.29744034340998]
  Mover is a new Deepfake detection model that exploits unspecific facial part inconsistencies. We propose a novel model with dual networks that utilize the pretrained encoder and masked autoencoder. Our experiments on standard benchmarks demonstrate that Mover is highly effective.
  arXiv  Detail & Related papers  (2023-03-03T06:57:22Z)
• Restricted Black-box Adversarial Attack Against DeepFake Face Swapping [70.82017781235535]
  We introduce a practical adversarial attack that does not require any queries to the facial image forgery model. Our method is built on a substitute model persuing for face reconstruction and then transfers adversarial examples from the substitute model directly to inaccessible black-box DeepFake models.
  arXiv  Detail & Related papers  (2022-04-26T14:36:06Z)
• Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection [112.96004727646115]
  We develop a method to detect face-manipulated videos using real talking faces. We show that our method achieves state-of-the-art performance on cross-manipulation generalisation and robustness experiments. Our results suggest that leveraging natural and unlabelled videos is a promising direction for the development of more robust face forgery detectors.
  arXiv  Detail & Related papers  (2022-01-18T17:14:54Z)
• Watch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion [82.06128362686445]
  We propose a multi-modal semantic forensic approach to handle both cheapfakes and visually persuasive deepfakes. We leverage the idea of attribution to learn person-specific biometric patterns that distinguish a given speaker from others. Unlike existing person-specific approaches, our method is also effective against attacks that focus on lip manipulation.
  arXiv  Detail & Related papers  (2021-12-21T01:57:04Z)
• FakeTransformer: Exposing Face Forgery From Spatial-Temporal Representation Modeled By Facial Pixel Variations [8.194624568473126]
  Face forgery can attack any target, which poses a new threat to personal privacy and property security. Inspired by the fact that the spatial coherence and temporal consistency of physiological signal are destroyed in the generated content, we attempt to find inconsistent patterns that can distinguish between real videos and synthetic videos.
  arXiv  Detail & Related papers  (2021-11-15T08:44:52Z)
• Face Forgery Detection by 3D Decomposition [72.22610063489248]
  We consider a face image as the production of the intervention of the underlying 3D geometry and the lighting environment. By disentangling the face image into 3D shape, common texture, identity texture, ambient light, and direct light, we find the devil lies in the direct light and the identity texture. We propose to utilize facial detail, which is the combination of direct light and identity texture, as the clue to detect the subtle forgery patterns.
  arXiv  Detail & Related papers  (2020-11-19T09:25:44Z)
• DeepFake Detection Based on the Discrepancy Between the Face and its Context [94.47879216590813]
  We propose a method for detecting face swapping and other identity manipulations in single images. Our approach involves two networks: (i) a face identification network that considers the face region bounded by a tight semantic segmentation, and (ii) a context recognition network that considers the face context. We describe a method which uses the recognition signals from our two networks to detect such discrepancies. Our method achieves state of the art results on the FaceForensics++, Celeb-DF-v2, and DFDC benchmarks for face manipulation detection, and even generalizes to detect fakes produced by unseen methods.
  arXiv  Detail & Related papers  (2020-08-27T17:04:46Z)
• Detecting Deep-Fake Videos from Appearance and Behavior [0.0]
  We describe a biometric-based forensic technique for detecting face-swap deep fakes. We show the efficacy of this approach across several large-scale video datasets.
  arXiv  Detail & Related papers  (2020-04-29T21:38:22Z)
• Video Face Manipulation Detection Through Ensemble of CNNs [17.051112469244778]
  We tackle the problem of face manipulation detection in video sequences targeting modern facial manipulation techniques. In particular, we study the ensembling of different trained Convolutional Neural Network (CNN) models. We show that combining these networks leads to promising face manipulation detection results on two publicly available datasets.
  arXiv  Detail & Related papers  (2020-04-16T14:19:40Z)
• Robust Facial Landmark Detection via Aggregation on Geometrically Manipulated Faces [32.391300491317445]
  We equip our method with the aggregation of manipulated face images. Small but carefully crafted geometric manipulation in the input domain can fool deep face recognition models. Our approach is demonstrated its superiority compared to the state-of-the-art method on benchmark datasets AFLW, 300-W, and COFW.
  arXiv  Detail & Related papers  (2020-01-07T16:43:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.