Facial expression and attributes recognition based on multi-task
learning of lightweight neural networks
- URL: http://arxiv.org/abs/2103.17107v1
- Date: Wed, 31 Mar 2021 14:21:04 GMT
- Title: Facial expression and attributes recognition based on multi-task
learning of lightweight neural networks
- Authors: Andrey V. Savchenko
- Abstract summary: We examine the multi-task training of lightweight convolutional neural networks for face identification and classification of facial attributes.
It is shown that it is still necessary to fine-tune these networks in order to predict facial expressions.
Several models are presented based on MobileNet, EfficientNet and RexNet architectures.
- Score: 9.162936410696409
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In this paper, we examine the multi-task training of lightweight
convolutional neural networks for face identification and classification of
facial attributes (age, gender, ethnicity) trained on cropped faces without
margins. It is shown that it is still necessary to fine-tune these networks in
order to predict facial expressions. Several models are presented based on
MobileNet, EfficientNet and RexNet architectures. It was experimentally
demonstrated that our models are characterized by the state-of-the-art emotion
classification accuracy on AffectNet dataset and near state-of-the-art results
in age, gender and race recognition for UTKFace dataset. Moreover, it is shown
that the usage of our neural network as a feature extractor of facial regions
in video frames and concatenation of several statistical functions (mean, max,
etc.) leads to 4.5\% higher accuracy than the previously known state-of-the-art
single models for AFEW and VGAF datasets from the EmotiW challenges. The models
and source code are publicly available at
https://github.com/HSE-asavchenko/face-emotion-recognition.
Related papers
- SwinFace: A Multi-task Transformer for Face Recognition, Expression
Recognition, Age Estimation and Attribute Estimation [60.94239810407917]
This paper presents a multi-purpose algorithm for simultaneous face recognition, facial expression recognition, age estimation, and face attribute estimation based on a single Swin Transformer.
To address the conflicts among multiple tasks, a Multi-Level Channel Attention (MLCA) module is integrated into each task-specific analysis.
Experiments show that the proposed model has a better understanding of the face and achieves excellent performance for all tasks.
arXiv Detail & Related papers (2023-08-22T15:38:39Z) - Research on facial expression recognition based on Multimodal data
fusion and neural network [2.5431493111705943]
The algorithm is based on the multimodal data, and it takes the facial image, the histogram of oriented gradient of the image and the facial landmarks as the input.
Experimental results show that, benefiting by the complementarity of multimodal data, the algorithm has a great improvement in accuracy, robustness and detection speed.
arXiv Detail & Related papers (2021-09-26T23:45:40Z) - Multi-Branch Deep Radial Basis Function Networks for Facial Emotion
Recognition [80.35852245488043]
We propose a CNN based architecture enhanced with multiple branches formed by radial basis function (RBF) units.
RBF units capture local patterns shared by similar instances using an intermediate representation.
We show it is the incorporation of local information what makes the proposed model competitive.
arXiv Detail & Related papers (2021-09-07T21:05:56Z) - Exploiting Emotional Dependencies with Graph Convolutional Networks for
Facial Expression Recognition [31.40575057347465]
This paper proposes a novel multi-task learning framework to recognize facial expressions in-the-wild.
A shared feature representation is learned for both discrete and continuous recognition in a MTL setting.
The results of our experiments show that our method outperforms the current state-of-the-art methods on discrete FER.
arXiv Detail & Related papers (2021-06-07T10:20:05Z) - Landmark-Aware and Part-based Ensemble Transfer Learning Network for
Facial Expression Recognition from Static images [0.5156484100374059]
Part-based Ensemble Transfer Learning network models how humans recognize facial expressions.
It consists of 5 sub-networks, in which each sub-network performs transfer learning from one of the five subsets of facial landmarks.
It requires only 3.28 $times$ $106$ FLOPS, which ensures computational efficiency for real-time deployment.
arXiv Detail & Related papers (2021-04-22T18:38:33Z) - Continuous Emotion Recognition with Spatiotemporal Convolutional Neural
Networks [82.54695985117783]
We investigate the suitability of state-of-the-art deep learning architectures for continuous emotion recognition using long video sequences captured in-the-wild.
We have developed and evaluated convolutional recurrent neural networks combining 2D-CNNs and long short term-memory units, and inflated 3D-CNN models, which are built by inflating the weights of a pre-trained 2D-CNN model during fine-tuning.
arXiv Detail & Related papers (2020-11-18T13:42:05Z) - Video-based Facial Expression Recognition using Graph Convolutional
Networks [57.980827038988735]
We introduce a Graph Convolutional Network (GCN) layer into a common CNN-RNN based model for video-based facial expression recognition.
We evaluate our method on three widely-used datasets, CK+, Oulu-CASIA and MMI, and also one challenging wild dataset AFEW8.0.
arXiv Detail & Related papers (2020-10-26T07:31:51Z) - The FaceChannelS: Strike of the Sequences for the AffWild 2 Challenge [80.07590100872548]
We show how our little model can predict affective information from the facial expression on the novel AffWild2 dataset.
In this paper, we present one more chapter of benchmarking different versions of the FaceChannel neural network.
arXiv Detail & Related papers (2020-10-04T12:00:48Z) - The FaceChannel: A Fast & Furious Deep Neural Network for Facial
Expression Recognition [71.24825724518847]
Current state-of-the-art models for automatic Facial Expression Recognition (FER) are based on very deep neural networks that are effective but rather expensive to train.
We formalize the FaceChannel, a light-weight neural network that has much fewer parameters than common deep neural networks.
We demonstrate how our model achieves a comparable, if not better, performance to the current state-of-the-art in FER.
arXiv Detail & Related papers (2020-09-15T09:25:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.