Watch Out for the Confusing Faces: Detecting Face Swapping with the Probability Distribution of Face Identification Models

• URL: http://arxiv.org/abs/2303.13131v1
• Date: Thu, 23 Mar 2023 09:33:10 GMT
• Title: Watch Out for the Confusing Faces: Detecting Face Swapping with the Probability Distribution of Face Identification Models
• Authors: Yuxuan Duan, Xuhong Zhang, Chuer Yu, Zonghui Wang, Shouling Ji, Wenzhi Chen
• Abstract summary: We propose a novel face swapping detection approach based on face identification probability distributions. IdP_FSD is specially designed for detecting swapped faces whose identities belong to a finite set. IdP_FSD exploits face swapping's common nature that the identity of swapped face combines that of two faces involved in swapping.
• Score: 37.49012763328351
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, face swapping has been developing rapidly and achieved a surprising reality, raising concerns about fake content. As a countermeasure, various detection approaches have been proposed and achieved promising performance. However, most existing detectors struggle to maintain performance on unseen face swapping methods and low-quality images. Apart from the generalization problem, current detection approaches have been shown vulnerable to evasion attacks crafted by detection-aware manipulators. Lack of robustness under adversary scenarios leaves threats for applying face swapping detection in real world. In this paper, we propose a novel face swapping detection approach based on face identification probability distributions, coined as IdP_FSD, to improve the generalization and robustness. IdP_FSD is specially designed for detecting swapped faces whose identities belong to a finite set, which is meaningful in real-world applications. Compared with previous general detection methods, we make use of the available real faces with concerned identities and require no fake samples for training. IdP_FSD exploits face swapping's common nature that the identity of swapped face combines that of two faces involved in swapping. We reflect this nature with the confusion of a face identification model and measure the confusion with the maximum value of the output probability distribution. What's more, to defend our detector under adversary scenarios, an attention-based finetuning scheme is proposed for the face identification models used in IdP_FSD. Extensive experiments show that the proposed IdP_FSD not only achieves high detection performance on different benchmark datasets and image qualities but also raises the bar for manipulators to evade the detection.

Related papers

• Imperceptible Face Forgery Attack via Adversarial Semantic Mask [59.23247545399068]
  We propose an Adversarial Semantic Mask Attack framework (ASMA) which can generate adversarial examples with good transferability and invisibility. Specifically, we propose a novel adversarial semantic mask generative model, which can constrain generated perturbations in local semantic regions for good stealthiness.
  arXiv  Detail & Related papers  (2024-06-16T10:38:11Z)
• Generalized Face Liveness Detection via De-spoofing Face Generator [58.7043386978171]
  Previous Face Anti-spoofing (FAS) works face the challenge of generalizing in unseen domains. We conduct an Anomalous cue Guided FAS (AG-FAS) method, which leverages real faces for improving model generalization via a De-spoofing Face Generator (DFG) We then propose an Anomalous cue Guided FAS feature extraction Network (AG-Net) to further improve the FAS feature generalization via a cross-attention transformer.
  arXiv  Detail & Related papers  (2024-01-17T06:59:32Z)
• DeepFidelity: Perceptual Forgery Fidelity Assessment for Deepfake Detection [67.3143177137102]
  Deepfake detection refers to detecting artificially generated or edited faces in images or videos. We propose a novel Deepfake detection framework named DeepFidelity to adaptively distinguish real and fake faces.
  arXiv  Detail & Related papers  (2023-12-07T07:19:45Z)
• Exploring Decision-based Black-box Attacks on Face Forgery Detection [53.181920529225906]
  Face forgery generation technologies generate vivid faces, which have raised public concerns about security and privacy. Although face forgery detection has successfully distinguished fake faces, recent studies have demonstrated that face forgery detectors are very vulnerable to adversarial examples.
  arXiv  Detail & Related papers  (2023-10-18T14:49:54Z)
• COMICS: End-to-end Bi-grained Contrastive Learning for Multi-face Forgery Detection [56.7599217711363]
  Face forgery recognition methods can only process one face at a time. Most face forgery recognition methods can only process one face at a time. We propose COMICS, an end-to-end framework for multi-face forgery detection.
  arXiv  Detail & Related papers  (2023-08-03T03:37:13Z)
• An Efficient Method for Face Quality Assessment on the Edge [1.7188280334580197]
  A practical approach on edge devices should prioritize these detection of identities according to their conformity to recognition. We propose a face quality score regression by just appending a single layer to a face landmark detection network. With almost no additional cost, face quality scores are obtained by training this single layer to regress recognition scores with surveillance like augmentations.
  arXiv  Detail & Related papers  (2022-07-19T18:29:43Z)
• RAF: Recursive Adversarial Attacks on Face Recognition Using Extremely Limited Queries [2.8532545355403123]
  Recent successful adversarial attacks on face recognition show that, despite the remarkable progress of face recognition models, they are still far behind the human intelligence for perception and recognition. In this paper, we propose automatic face warping which needs extremely limited number of queries to fool the target model. We evaluate the robustness of proposed method in the decision-based black-box attack setting.
  arXiv  Detail & Related papers  (2022-07-04T00:22:45Z)
• On the (Limited) Generalization of MasterFace Attacks and Its Relation to the Capacity of Face Representations [11.924504853735645]
  We study the generalizability of MasterFace attacks in empirical and theoretical investigations. We estimate the face capacity and the maximum MasterFace coverage under the assumption that identities in the face space are well separated. We conclude that MasterFaces should not be seen as a threat to face recognition systems but as a tool to enhance the robustness of face recognition models.
  arXiv  Detail & Related papers  (2022-03-23T13:02:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.