Joint Deep Learning of Facial Expression Synthesis and Recognition
- URL: http://arxiv.org/abs/2002.02194v1
- Date: Thu, 6 Feb 2020 10:56:00 GMT
- Title: Joint Deep Learning of Facial Expression Synthesis and Recognition
- Authors: Yan Yan, Ying Huang, Si Chen, Chunhua Shen, Hanzi Wang
- Abstract summary: We propose a novel joint deep learning of facial expression synthesis and recognition method for effective FER.
The proposed method involves a two-stage learning procedure. Firstly, a facial expression synthesis generative adversarial network (FESGAN) is pre-trained to generate facial images with different facial expressions.
In order to alleviate the problem of data bias between the real images and the synthetic images, we propose an intra-class loss with a novel real data-guided back-propagation (RDBP) algorithm.
- Score: 97.19528464266824
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recently, deep learning based facial expression recognition (FER) methods
have attracted considerable attention and they usually require large-scale
labelled training data. Nonetheless, the publicly available facial expression
databases typically contain a small amount of labelled data. In this paper, to
overcome the above issue, we propose a novel joint deep learning of facial
expression synthesis and recognition method for effective FER. More
specifically, the proposed method involves a two-stage learning procedure.
Firstly, a facial expression synthesis generative adversarial network (FESGAN)
is pre-trained to generate facial images with different facial expressions. To
increase the diversity of the training images, FESGAN is elaborately designed
to generate images with new identities from a prior distribution. Secondly, an
expression recognition network is jointly learned with the pre-trained FESGAN
in a unified framework. In particular, the classification loss computed from
the recognition network is used to simultaneously optimize the performance of
both the recognition network and the generator of FESGAN. Moreover, in order to
alleviate the problem of data bias between the real images and the synthetic
images, we propose an intra-class loss with a novel real data-guided
back-propagation (RDBP) algorithm to reduce the intra-class variations of
images from the same class, which can significantly improve the final
performance. Extensive experimental results on public facial expression
databases demonstrate the superiority of the proposed method compared with
several state-of-the-art FER methods.
Related papers
- CemiFace: Center-based Semi-hard Synthetic Face Generation for Face Recognition [33.17771044475894]
We show that face images with certain degree of similarities to their identity centers show great effectiveness in the performance of trained face recognition models.
Inspired by this, we propose a novel diffusion-based approach (namely Center-based Semi-hard Synthetic Face Generation) which produces facial samples with various levels of similarity to the subject center.
arXiv Detail & Related papers (2024-09-27T16:11:30Z) - Alleviating Catastrophic Forgetting in Facial Expression Recognition with Emotion-Centered Models [49.3179290313959]
The proposed method, emotion-centered generative replay (ECgr), tackles this challenge by integrating synthetic images from generative adversarial networks.
ECgr incorporates a quality assurance algorithm to ensure the fidelity of generated images.
The experimental results on four diverse facial expression datasets demonstrate that incorporating images generated by our pseudo-rehearsal method enhances training on the targeted dataset and the source dataset.
arXiv Detail & Related papers (2024-04-18T15:28:34Z) - Research on facial expression recognition based on Multimodal data
fusion and neural network [2.5431493111705943]
The algorithm is based on the multimodal data, and it takes the facial image, the histogram of oriented gradient of the image and the facial landmarks as the input.
Experimental results show that, benefiting by the complementarity of multimodal data, the algorithm has a great improvement in accuracy, robustness and detection speed.
arXiv Detail & Related papers (2021-09-26T23:45:40Z) - Self-supervised Contrastive Learning of Multi-view Facial Expressions [9.949781365631557]
Facial expression recognition (FER) has emerged as an important component of human-computer interaction systems.
We propose Contrastive Learning of Multi-view facial Expressions (CL-MEx) to exploit facial images captured simultaneously from different angles towards FER.
arXiv Detail & Related papers (2021-08-15T11:23:34Z) - Identity-Aware CycleGAN for Face Photo-Sketch Synthesis and Recognition [61.87842307164351]
We first propose an Identity-Aware CycleGAN (IACycleGAN) model that applies a new perceptual loss to supervise the image generation network.
It improves CycleGAN on photo-sketch synthesis by paying more attention to the synthesis of key facial regions, such as eyes and nose.
We develop a mutual optimization procedure between the synthesis model and the recognition model, which iteratively synthesizes better images by IACycleGAN.
arXiv Detail & Related papers (2021-03-30T01:30:08Z) - Multi-Margin based Decorrelation Learning for Heterogeneous Face
Recognition [90.26023388850771]
This paper presents a deep neural network approach to extract decorrelation representations in a hyperspherical space for cross-domain face images.
The proposed framework can be divided into two components: heterogeneous representation network and decorrelation representation learning.
Experimental results on two challenging heterogeneous face databases show that our approach achieves superior performance on both verification and recognition tasks.
arXiv Detail & Related papers (2020-05-25T07:01:12Z) - Deep Face Super-Resolution with Iterative Collaboration between
Attentive Recovery and Landmark Estimation [92.86123832948809]
We propose a deep face super-resolution (FSR) method with iterative collaboration between two recurrent networks.
In each recurrent step, the recovery branch utilizes the prior knowledge of landmarks to yield higher-quality images.
A new attentive fusion module is designed to strengthen the guidance of landmark maps.
arXiv Detail & Related papers (2020-03-29T16:04:48Z) - Deep Multi-Facial Patches Aggregation Network For Facial Expression
Recognition [5.735035463793008]
We propose an approach for Facial Expressions Recognition (FER) based on a deep multi-facial patches aggregation network.
Deep features are learned from facial patches using deep sub-networks and aggregated within one deep architecture for expression classification.
arXiv Detail & Related papers (2020-02-20T17:57:06Z) - Exploiting Semantics for Face Image Deblurring [121.44928934662063]
We propose an effective and efficient face deblurring algorithm by exploiting semantic cues via deep convolutional neural networks.
We incorporate face semantic labels as input priors and propose an adaptive structural loss to regularize facial local structures.
The proposed method restores sharp images with more accurate facial features and details.
arXiv Detail & Related papers (2020-01-19T13:06:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.