Detection and Localization of Facial Expression Manipulations
- URL: http://arxiv.org/abs/2103.08134v1
- Date: Mon, 15 Mar 2021 04:35:56 GMT
- Title: Detection and Localization of Facial Expression Manipulations
- Authors: Ghazal Mazaheri, Amit K. Roy-Chowdhury
- Abstract summary: We propose a framework that is able to detect manipulations in facial expression using a close combination of facial expression recognition and image manipulation methods.
We show that, on the Face2Face dataset, where there is abundant expression manipulation, our method achieves over 3% higher accuracy for both classification and localization of manipulations.
- Score: 44.52966548652561
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Concern regarding the wide-spread use of fraudulent images/videos in social
media necessitates precise detection of such fraud. The importance of facial
expressions in communication is widely known, and adversarial attacks often
focus on manipulating the expression related features. Thus, it is important to
develop methods that can detect manipulations in facial expressions, and
localize the manipulated regions. To address this problem, we propose a
framework that is able to detect manipulations in facial expression using a
close combination of facial expression recognition and image manipulation
methods. With the addition of feature maps extracted from the facial expression
recognition framework, our manipulation detector is able to localize the
manipulated region. We show that, on the Face2Face dataset, where there is
abundant expression manipulation, our method achieves over 3% higher accuracy
for both classification and localization of manipulations compared to
state-of-the-art methods. In addition, results on the NeuralTextures dataset
where the facial expressions corresponding to the mouth regions have been
modified, show 2% higher accuracy in both classification and localization of
manipulation. We demonstrate that the method performs at-par with the
state-of-the-art methods in cases where the expression is not manipulated, but
rather the identity is changed, thus ensuring generalizability of the approach.
Related papers
- Learning Expressive And Generalizable Motion Features For Face Forgery
Detection [52.54404879581527]
We propose an effective sequence-based forgery detection framework based on an existing video classification method.
To make the motion features more expressive for manipulation detection, we propose an alternative motion consistency block.
We make a general video classification network achieve promising results on three popular face forgery datasets.
arXiv Detail & Related papers (2024-03-08T09:25:48Z) - Collaborative Feature Learning for Fine-grained Facial Forgery Detection
and Segmentation [56.73855202368894]
Previous work related to forgery detection mostly focuses on the entire faces.
Recent forgery methods have developed to edit important facial components while maintaining others unchanged.
We propose a collaborative feature learning approach to simultaneously detect manipulation and segment the falsified components.
arXiv Detail & Related papers (2023-04-17T08:49:11Z) - Face Forgery Detection Based on Facial Region Displacement Trajectory
Series [10.338298543908339]
We develop a method for detecting manipulated videos based on the trajectory of the facial region displacement.
This information was used to construct a network for exposing multidimensional artifacts in the trajectory sequences of manipulated videos.
arXiv Detail & Related papers (2022-12-07T14:47:54Z) - Detect and Locate: A Face Anti-Manipulation Approach with Semantic and
Noise-level Supervision [67.73180660609844]
We propose a conceptually simple but effective method to efficiently detect forged faces in an image.
The proposed scheme relies on a segmentation map that delivers meaningful high-level semantic information clues about the image.
The proposed model achieves state-of-the-art detection accuracy and remarkable localization performance.
arXiv Detail & Related papers (2021-07-13T02:59:31Z) - Progressive Spatio-Temporal Bilinear Network with Monte Carlo Dropout
for Landmark-based Facial Expression Recognition with Uncertainty Estimation [93.73198973454944]
The performance of our method is evaluated on three widely used datasets.
It is comparable to that of video-based state-of-the-art methods while it has much less complexity.
arXiv Detail & Related papers (2021-06-08T13:40:30Z) - DeepFake Detection Based on the Discrepancy Between the Face and its
Context [94.47879216590813]
We propose a method for detecting face swapping and other identity manipulations in single images.
Our approach involves two networks: (i) a face identification network that considers the face region bounded by a tight semantic segmentation, and (ii) a context recognition network that considers the face context.
We describe a method which uses the recognition signals from our two networks to detect such discrepancies.
Our method achieves state of the art results on the FaceForensics++, Celeb-DF-v2, and DFDC benchmarks for face manipulation detection, and even generalizes to detect fakes produced by unseen methods.
arXiv Detail & Related papers (2020-08-27T17:04:46Z) - Fake face detection via adaptive manipulation traces extraction network [9.892936175042939]
We propose an adaptive manipulation traces extraction network (AMTEN) to suppress image content and highlight manipulation traces.
AMTEN exploits an adaptive convolution layer to predict manipulation traces in the image, which are reused in subsequent layers to maximize manipulation artifacts.
When detecting fake face images generated by various FIM techniques, AMTENnet achieves an average accuracy up to 98.52%, which outperforms the state-of-the-art works.
arXiv Detail & Related papers (2020-05-11T09:16:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.