Neuromorphic Valence and Arousal Estimation

• URL: http://arxiv.org/abs/2401.16058v1
• Date: Mon, 29 Jan 2024 11:13:18 GMT
• Title: Neuromorphic Valence and Arousal Estimation
• Authors: Lorenzo Berlincioni, Luca Cultrera, Federico Becattini, Alberto Del Bimbo
• Abstract summary: We use neuromorphic data to predict emotional states from faces. We demonstrate that our trained models can still yield state-of-the-art results. In the paper we propose several alternative models to solve the task, both frame-based and video-based.
• Score: 28.793519320598865
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recognizing faces and their underlying emotions is an important aspect of biometrics. In fact, estimating emotional states from faces has been tackled from several angles in the literature. In this paper, we follow the novel route of using neuromorphic data to predict valence and arousal values from faces. Due to the difficulty of gathering event-based annotated videos, we leverage an event camera simulator to create the neuromorphic counterpart of an existing RGB dataset. We demonstrate that not only training models on simulated data can still yield state-of-the-art results in valence-arousal estimation, but also that our trained models can be directly applied to real data without further training to address the downstream task of emotion recognition. In the paper we propose several alternative models to solve the task, both frame-based and video-based.

Related papers

• Data Augmentation and Transfer Learning Approaches Applied to Facial Expressions Recognition [0.3481985817302898]
  We propose a novel data augmentation technique that improves the performances in the recognition task. We build from scratch GAN models able to generate new synthetic images for each emotion type. On the augmented datasets we fine tune pretrained convolutional neural networks with different architectures.
  arXiv  Detail & Related papers  (2024-02-15T14:46:03Z)
• Learning Human Action Recognition Representations Without Real Humans [66.61527869763819]
  We present a benchmark that leverages real-world videos with humans removed and synthetic data containing virtual humans to pre-train a model. We then evaluate the transferability of the representation learned on this data to a diverse set of downstream action recognition benchmarks. Our approach outperforms previous baselines by up to 5%.
  arXiv  Detail & Related papers  (2023-11-10T18:38:14Z)
• Neural Implicit Representations for Physical Parameter Inference from a Single Video [49.766574469284485]
  We propose to combine neural implicit representations for appearance modeling with neural ordinary differential equations (ODEs) for modelling physical phenomena. Our proposed model combines several unique advantages: (i) Contrary to existing approaches that require large training datasets, we are able to identify physical parameters from only a single video. The use of neural implicit representations enables the processing of high-resolution videos and the synthesis of photo-realistic images.
  arXiv  Detail & Related papers  (2022-04-29T11:55:35Z)
• A Robust Framework for Deep Learning Approaches to Facial Emotion Recognition and Evaluation [0.17398560678845074]
  We propose a framework in which models developed for FER can be compared and contrasted against one another. A lightweight convolutional neural network is trained on the AffectNet dataset. A web application is developed and deployed with our proposed framework as a proof of concept.
  arXiv  Detail & Related papers  (2022-01-30T02:10:01Z)
• Simulated Adversarial Testing of Face Recognition Models [53.10078734154151]
  We propose a framework for learning how to test machine learning algorithms using simulators in an adversarial manner. We are the first to show that weaknesses of models trained on real data can be discovered using simulated samples.
  arXiv  Detail & Related papers  (2021-06-08T17:58:10Z)
• Continuous Emotion Recognition with Spatiotemporal Convolutional Neural Networks [82.54695985117783]
  We investigate the suitability of state-of-the-art deep learning architectures for continuous emotion recognition using long video sequences captured in-the-wild. We have developed and evaluated convolutional recurrent neural networks combining 2D-CNNs and long short term-memory units, and inflated 3D-CNN models, which are built by inflating the weights of a pre-trained 2D-CNN model during fine-tuning.
  arXiv  Detail & Related papers  (2020-11-18T13:42:05Z)
• The FaceChannel: A Fast & Furious Deep Neural Network for Facial Expression Recognition [71.24825724518847]
  Current state-of-the-art models for automatic Facial Expression Recognition (FER) are based on very deep neural networks that are effective but rather expensive to train. We formalize the FaceChannel, a light-weight neural network that has much fewer parameters than common deep neural networks. We demonstrate how our model achieves a comparable, if not better, performance to the current state-of-the-art in FER.
  arXiv  Detail & Related papers  (2020-09-15T09:25:37Z)
• Deformation-aware Unpaired Image Translation for Pose Estimation on Laboratory Animals [56.65062746564091]
  We aim to capture the pose of neuroscience model organisms, without using any manual supervision, to study how neural circuits orchestrate behaviour. Our key contribution is the explicit and independent modeling of appearance, shape and poses in an unpaired image translation framework. We demonstrate improved pose estimation accuracy on Drosophila melanogaster (fruit fly), Caenorhabditis elegans (worm) and Danio rerio (zebrafish)
  arXiv  Detail & Related papers  (2020-01-23T15:34:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.