Mining Generalized Features for Detecting AI-Manipulated Fake Faces

• URL: http://arxiv.org/abs/2010.14129v1
• Date: Tue, 27 Oct 2020 08:41:16 GMT
• Title: Mining Generalized Features for Detecting AI-Manipulated Fake Faces
• Authors: Yang Yu, Rongrong Ni and Yao Zhao
• Abstract summary: We propose a novel framework that focuses on mining intrinsic features and eliminating the distribution bias to improve the generalization ability. We evaluate the proposed framework on four categories of fake faces datasets with the most popular and state-of-the-art manipulation techniques.
• Score: 39.86126596985567
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, AI-manipulated face techniques have developed rapidly and constantly, which has raised new security issues in society. Although existing detection methods consider different categories of fake faces, the performance on detecting the fake faces with "unseen" manipulation techniques is still poor due to the distribution bias among cross-manipulation techniques. To solve this problem, we propose a novel framework that focuses on mining intrinsic features and further eliminating the distribution bias to improve the generalization ability. Firstly, we focus on mining the intrinsic clues in the channel difference image (CDI) and spectrum image (SI) from the camera imaging process and the indispensable step in AI manipulation process. Then, we introduce the Octave Convolution (OctConv) and an attention-based fusion module to effectively and adaptively mine intrinsic features from CDI and SI. Finally, we design an alignment module to eliminate the bias of manipulation techniques to obtain a more generalized detection framework. We evaluate the proposed framework on four categories of fake faces datasets with the most popular and state-of-the-art manipulation techniques, and achieve very competitive performances. To further verify the generalization ability of the proposed framework, we conduct experiments on cross-manipulation techniques, and the results show the advantages of our method.

Related papers

• Understanding and Improving Training-Free AI-Generated Image Detections with Vision Foundation Models [68.90917438865078]
  Deepfake techniques for facial synthesis and editing pose serious risks for generative models. In this paper, we investigate how detection performance varies across model backbones, types, and datasets. We introduce Contrastive Blur, which enhances performance on facial images, and MINDER, which addresses noise type bias, balancing performance across domains.
  arXiv  Detail & Related papers  (2024-11-28T13:04:45Z)
• Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detection [58.87142367781417]
  A naively trained detector tends to favor overfitting to the limited and monotonous fake patterns, causing the feature space to become highly constrained and low-ranked. One potential remedy is incorporating the pre-trained knowledge within the vision foundation models to expand the feature space. By freezing the principal components and adapting only the remained components, we preserve the pre-trained knowledge while learning forgery-related patterns.
  arXiv  Detail & Related papers  (2024-11-23T19:10:32Z)
• UniForensics: Face Forgery Detection via General Facial Representation [60.5421627990707]
  High-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization. We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video network, with a meta-functional face classification for enriched facial representation.
  arXiv  Detail & Related papers  (2024-07-26T20:51:54Z)
• A visualization method for data domain changes in CNN networks and the optimization method for selecting thresholds in classification tasks [1.1118946307353794]
  Face Anti-Spoofing (FAS) has played a crucial role in preserving the security of face recognition technology. With the rise of counterfeit face generation techniques, the challenge posed by digitally edited faces to face anti-spoofing is escalating. We propose a visualization method that intuitively reflects the training outcomes of models by visualizing the prediction results on datasets.
  arXiv  Detail & Related papers  (2024-04-19T03:12:17Z)
• Detect and Locate: A Face Anti-Manipulation Approach with Semantic and Noise-level Supervision [67.73180660609844]
  We propose a conceptually simple but effective method to efficiently detect forged faces in an image. The proposed scheme relies on a segmentation map that delivers meaningful high-level semantic information clues about the image. The proposed model achieves state-of-the-art detection accuracy and remarkable localization performance.
  arXiv  Detail & Related papers  (2021-07-13T02:59:31Z)
• Towards Generalizable and Robust Face Manipulation Detection via Bag-of-local-feature [55.47546606878931]
  We propose a novel method for face manipulation detection, which can improve the generalization ability and robustness by bag-of-local-feature. Specifically, we extend Transformers using bag-of-feature approach to encode inter-patch relationships, allowing it to learn local forgery features without any explicit supervision.
  arXiv  Detail & Related papers  (2021-03-14T12:50:48Z)
• Attentive Semantic Exploring for Manipulated Face Detection [12.635690519021555]
  We find that segmenting images into semantic fragments could be effective, as discriminative defects and distortions are closely related to such fragments. We propose a novel manipulated face detection method based on Multilevel Facial Semantic and Cascade Attention Mechanism.
  arXiv  Detail & Related papers  (2020-05-06T17:08:56Z)
• DeepFake Detection by Analyzing Convolutional Traces [0.0]
  We focus on the analysis of Deepfakes of human faces with the objective of creating a new detection method. The proposed technique, by means of an Expectation Maximization (EM) algorithm, extracts a set of local features specifically addressed to model the underlying convolutional generative process. Results demonstrated the effectiveness of the technique in distinguishing the different architectures and the corresponding generation process.
  arXiv  Detail & Related papers  (2020-04-22T09:02:55Z)
• FakeLocator: Robust Localization of GAN-Based Face Manipulations [19.233930372590226]
  We propose a novel approach, termed FakeLocator, to obtain high localization accuracy, at full resolution, on manipulated facial images. This is the very first attempt to solve the GAN-based fake localization problem with a gray-scale fakeness map. Experimental results on popular FaceForensics++, DFFD datasets and seven different state-of-the-art GAN-based face generation methods have shown the effectiveness of our method.
  arXiv  Detail & Related papers  (2020-01-27T06:15:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.