Emotion recognition in talking-face videos using persistent entropy and neural networks

• URL: http://arxiv.org/abs/2110.13571v1
• Date: Tue, 26 Oct 2021 11:08:56 GMT
• Title: Emotion recognition in talking-face videos using persistent entropy and neural networks
• Authors: Eduardo Paluzo-Hidalgo, Guillermo Aguirre-Carrazana, Rocio Gonzalez-Diaz
• Abstract summary: We use persistent entropy and neural networks as main tools to recognise and classify emotions from talking-face videos. We prove that small changes in the video produce small changes in the signature. These topological signatures are used to feed a neural network to distinguish between the following emotions: neutral, calm, happy, sad, angry, fearful, disgust, and surprised.
• Score: 0.5156484100374059
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The automatic recognition of a person's emotional state has become a very active research field that involves scientists specialized in different areas such as artificial intelligence, computer vision or psychology, among others. Our main objective in this work is to develop a novel approach, using persistent entropy and neural networks as main tools, to recognise and classify emotions from talking-face videos. Specifically, we combine audio-signal and image-sequence information to compute a topology signature(a 9-dimensional vector) for each video. We prove that small changes in the video produce small changes in the signature. These topological signatures are used to feed a neural network to distinguish between the following emotions: neutral, calm, happy, sad, angry, fearful, disgust, and surprised. The results reached are promising and competitive, beating the performance reached in other state-of-the-art works found in the literature.

Related papers

• Authentic Emotion Mapping: Benchmarking Facial Expressions in Real News [21.707761612280304]
  We present a novel benchmark for Emotion Recognition using facial landmarks extracted from realistic news videos. Traditional methods relying on RGB images are resource-intensive, whereas our approach with Facial Landmark Emotion Recognition (FLER) offers a simplified yet effective alternative.
  arXiv  Detail & Related papers  (2024-04-21T00:14:03Z)
• Emotion Analysis on EEG Signal Using Machine Learning and Neural Network [0.0]
  The main purpose of this study is to improve ways to improve emotion recognition performance using brain signals. Various approaches to human-machine interaction technologies have been ongoing for a long time, and in recent years, researchers have had great success in automatically understanding emotion using brain signals.
  arXiv  Detail & Related papers  (2023-07-09T09:50:34Z)
• Multi-Cue Adaptive Emotion Recognition Network [4.570705738465714]
  We propose a new deep learning approach for emotion recognition based on adaptive multi-cues. We compare the proposed approach with the state-of-art approaches in the CAER-S dataset.
  arXiv  Detail & Related papers  (2021-11-03T15:08:55Z)
• SOLVER: Scene-Object Interrelated Visual Emotion Reasoning Network [83.27291945217424]
  We propose a novel Scene-Object interreLated Visual Emotion Reasoning network (SOLVER) to predict emotions from images. To mine the emotional relationships between distinct objects, we first build up an Emotion Graph based on semantic concepts and visual features. We also design a Scene-Object Fusion Module to integrate scenes and objects, which exploits scene features to guide the fusion process of object features with the proposed scene-based attention mechanism.
  arXiv  Detail & Related papers  (2021-10-24T02:41:41Z)
• Stimuli-Aware Visual Emotion Analysis [75.68305830514007]
  We propose a stimuli-aware visual emotion analysis (VEA) method consisting of three stages, namely stimuli selection, feature extraction and emotion prediction. To the best of our knowledge, it is the first time to introduce stimuli selection process into VEA in an end-to-end network. Experiments demonstrate that the proposed method consistently outperforms the state-of-the-art approaches on four public visual emotion datasets.
  arXiv  Detail & Related papers  (2021-09-04T08:14:52Z)
• Leveraging Recent Advances in Deep Learning for Audio-Visual Emotion Recognition [2.1485350418225244]
  Spontaneous multi-modal emotion recognition has been extensively studied for human behavior analysis. We propose a new deep learning-based approach for audio-visual emotion recognition.
  arXiv  Detail & Related papers  (2021-03-16T15:49:15Z)
• Emotion pattern detection on facial videos using functional statistics [62.997667081978825]
  We propose a technique based on Functional ANOVA to extract significant patterns of face muscles movements. We determine if there are time-related differences on expressions among emotional groups by using a functional F-test.
  arXiv  Detail & Related papers  (2021-03-01T08:31:08Z)
• Continuous Emotion Recognition with Spatiotemporal Convolutional Neural Networks [82.54695985117783]
  We investigate the suitability of state-of-the-art deep learning architectures for continuous emotion recognition using long video sequences captured in-the-wild. We have developed and evaluated convolutional recurrent neural networks combining 2D-CNNs and long short term-memory units, and inflated 3D-CNN models, which are built by inflating the weights of a pre-trained 2D-CNN model during fine-tuning.
  arXiv  Detail & Related papers  (2020-11-18T13:42:05Z)
• Facial Emotion Recognition with Noisy Multi-task Annotations [88.42023952684052]
  We introduce a new problem of facial emotion recognition with noisy multi-task annotations. For this new problem, we suggest a formulation from the point of joint distribution match view. We exploit a new method to enable the emotion prediction and the joint distribution learning.
  arXiv  Detail & Related papers  (2020-10-19T20:39:37Z)
• Emotional Video to Audio Transformation Using Deep Recurrent Neural Networks and a Neuro-Fuzzy System [8.900866276512364]
  Current approaches overlook the video's emotional characteristics in the music generation step. We propose a novel hybrid deep neural network that uses an Adaptive Neuro-Fuzzy Inference System to predict a video's emotion. Our model can effectively generate audio that matches the scene eliciting a similar emotion from the viewer in both datasets.
  arXiv  Detail & Related papers  (2020-04-05T07:18:28Z)
• An End-to-End Visual-Audio Attention Network for Emotion Recognition in User-Generated Videos [64.91614454412257]
  We propose to recognize video emotions in an end-to-end manner based on convolutional neural networks (CNNs) Specifically, we develop a deep Visual-Audio Attention Network (VAANet), a novel architecture that integrates spatial, channel-wise, and temporal attentions into a visual 3D CNN and temporal attentions into an audio 2D CNN.
  arXiv  Detail & Related papers  (2020-02-12T15:33:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.