Mining Generalized Features for Detecting AI-Manipulated Fake Faces
- URL: http://arxiv.org/abs/2010.14129v1
- Date: Tue, 27 Oct 2020 08:41:16 GMT
- Title: Mining Generalized Features for Detecting AI-Manipulated Fake Faces
- Authors: Yang Yu, Rongrong Ni and Yao Zhao
- Abstract summary: We propose a novel framework that focuses on mining intrinsic features and eliminating the distribution bias to improve the generalization ability.
We evaluate the proposed framework on four categories of fake faces datasets with the most popular and state-of-the-art manipulation techniques.
- Score: 39.86126596985567
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recently, AI-manipulated face techniques have developed rapidly and
constantly, which has raised new security issues in society. Although existing
detection methods consider different categories of fake faces, the performance
on detecting the fake faces with "unseen" manipulation techniques is still poor
due to the distribution bias among cross-manipulation techniques. To solve this
problem, we propose a novel framework that focuses on mining intrinsic features
and further eliminating the distribution bias to improve the generalization
ability. Firstly, we focus on mining the intrinsic clues in the channel
difference image (CDI) and spectrum image (SI) from the camera imaging process
and the indispensable step in AI manipulation process. Then, we introduce the
Octave Convolution (OctConv) and an attention-based fusion module to
effectively and adaptively mine intrinsic features from CDI and SI. Finally, we
design an alignment module to eliminate the bias of manipulation techniques to
obtain a more generalized detection framework. We evaluate the proposed
framework on four categories of fake faces datasets with the most popular and
state-of-the-art manipulation techniques, and achieve very competitive
performances. To further verify the generalization ability of the proposed
framework, we conduct experiments on cross-manipulation techniques, and the
results show the advantages of our method.
Related papers
- MFCLIP: Multi-modal Fine-grained CLIP for Generalizable Diffusion Face Forgery Detection [64.29452783056253]
The rapid development of photo-realistic face generation methods has raised significant concerns in society and academia.
Although existing approaches mainly capture face forgery patterns using image modality, other modalities like fine-grained noises and texts are not fully explored.
We propose a novel multi-modal fine-grained CLIP (MFCLIP) model, which mines comprehensive and fine-grained forgery traces across image-noise modalities.
arXiv Detail & Related papers (2024-09-15T13:08:59Z) - UniForensics: Face Forgery Detection via General Facial Representation [60.5421627990707]
High-level semantic features are less susceptible to perturbations and not limited to forgery-specific artifacts, thus having stronger generalization.
We introduce UniForensics, a novel deepfake detection framework that leverages a transformer-based video network, with a meta-functional face classification for enriched facial representation.
arXiv Detail & Related papers (2024-07-26T20:51:54Z) - A visualization method for data domain changes in CNN networks and the optimization method for selecting thresholds in classification tasks [1.1118946307353794]
Face Anti-Spoofing (FAS) has played a crucial role in preserving the security of face recognition technology.
With the rise of counterfeit face generation techniques, the challenge posed by digitally edited faces to face anti-spoofing is escalating.
We propose a visualization method that intuitively reflects the training outcomes of models by visualizing the prediction results on datasets.
arXiv Detail & Related papers (2024-04-19T03:12:17Z) - GenFace: A Large-Scale Fine-Grained Face Forgery Benchmark and Cross Appearance-Edge Learning [50.7702397913573]
The rapid advancement of photorealistic generators has reached a critical juncture where the discrepancy between authentic and manipulated images is increasingly indistinguishable.
Although there have been a number of publicly available face forgery datasets, the forgery faces are mostly generated using GAN-based synthesis technology.
We propose a large-scale, diverse, and fine-grained high-fidelity dataset, namely GenFace, to facilitate the advancement of deepfake detection.
arXiv Detail & Related papers (2024-02-03T03:13:50Z) - Detect and Locate: A Face Anti-Manipulation Approach with Semantic and
Noise-level Supervision [67.73180660609844]
We propose a conceptually simple but effective method to efficiently detect forged faces in an image.
The proposed scheme relies on a segmentation map that delivers meaningful high-level semantic information clues about the image.
The proposed model achieves state-of-the-art detection accuracy and remarkable localization performance.
arXiv Detail & Related papers (2021-07-13T02:59:31Z) - Towards Generalizable and Robust Face Manipulation Detection via
Bag-of-local-feature [55.47546606878931]
We propose a novel method for face manipulation detection, which can improve the generalization ability and robustness by bag-of-local-feature.
Specifically, we extend Transformers using bag-of-feature approach to encode inter-patch relationships, allowing it to learn local forgery features without any explicit supervision.
arXiv Detail & Related papers (2021-03-14T12:50:48Z) - Attentive Semantic Exploring for Manipulated Face Detection [12.635690519021555]
We find that segmenting images into semantic fragments could be effective, as discriminative defects and distortions are closely related to such fragments.
We propose a novel manipulated face detection method based on Multilevel Facial Semantic and Cascade Attention Mechanism.
arXiv Detail & Related papers (2020-05-06T17:08:56Z) - DeepFake Detection by Analyzing Convolutional Traces [0.0]
We focus on the analysis of Deepfakes of human faces with the objective of creating a new detection method.
The proposed technique, by means of an Expectation Maximization (EM) algorithm, extracts a set of local features specifically addressed to model the underlying convolutional generative process.
Results demonstrated the effectiveness of the technique in distinguishing the different architectures and the corresponding generation process.
arXiv Detail & Related papers (2020-04-22T09:02:55Z) - FakeLocator: Robust Localization of GAN-Based Face Manipulations [19.233930372590226]
We propose a novel approach, termed FakeLocator, to obtain high localization accuracy, at full resolution, on manipulated facial images.
This is the very first attempt to solve the GAN-based fake localization problem with a gray-scale fakeness map.
Experimental results on popular FaceForensics++, DFFD datasets and seven different state-of-the-art GAN-based face generation methods have shown the effectiveness of our method.
arXiv Detail & Related papers (2020-01-27T06:15:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.