Emotion Recognition on large video dataset based on Convolutional Feature Extractor and Recurrent Neural Network

• URL: http://arxiv.org/abs/2006.11168v1
• Date: Fri, 19 Jun 2020 14:54:13 GMT
• Title: Emotion Recognition on large video dataset based on Convolutional Feature Extractor and Recurrent Neural Network
• Authors: Denis Rangulov, Muhammad Fahim
• Abstract summary: Our model combines convolutional neural network (CNN) with recurrent neural network (RNN) to predict dimensional emotions on video data. Experiments are performed on publicly available datasets including the largest modern Aff-Wild2 database.
• Score: 0.2855485723554975
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For many years, the emotion recognition task has remained one of the most interesting and important problems in the field of human-computer interaction. In this study, we consider the emotion recognition task as a classification as well as a regression task by processing encoded emotions in different datasets using deep learning models. Our model combines convolutional neural network (CNN) with recurrent neural network (RNN) to predict dimensional emotions on video data. At the first step, CNN extracts feature vectors from video frames. In the second step, we fed these feature vectors to train RNN for exploiting the temporal dynamics of video. Furthermore, we analyzed how each neural network contributes to the system's overall performance. The experiments are performed on publicly available datasets including the largest modern Aff-Wild2 database. It contains over sixty hours of video data. We discovered the problem of overfitting of the model on an unbalanced dataset with an illustrative example using confusion matrices. The problem is solved by downsampling technique to balance the dataset. By significantly decreasing training data, we balance the dataset, thereby, the overall performance of the model is improved. Hence, the study qualitatively describes the abilities of deep learning models exploring enough amount of data to predict facial emotions. Our proposed method is implemented using Tensorflow Keras.

Related papers

• Dynamic Analysis and an Eigen Initializer for Recurrent Neural Networks [0.0]
  We study the dynamics of the hidden state in recurrent neural networks. We propose a new perspective to analyze the hidden state space based on an eigen decomposition of the weight matrix. We provide an explanation for long-term dependency based on the eigen analysis.
  arXiv  Detail & Related papers  (2023-07-28T17:14:58Z)
• Progressive Fourier Neural Representation for Sequential Video Compilation [75.43041679717376]
  Motivated by continual learning, this work investigates how to accumulate and transfer neural implicit representations for multiple complex video data over sequential encoding sessions. We propose a novel method, Progressive Fourier Neural Representation (PFNR), that aims to find an adaptive and compact sub-module in Fourier space to encode videos in each training session. We validate our PFNR method on the UVG8/17 and DAVIS50 video sequence benchmarks and achieve impressive performance gains over strong continual learning baselines.
  arXiv  Detail & Related papers  (2023-06-20T06:02:19Z)
• Diffused Redundancy in Pre-trained Representations [98.55546694886819]
  We take a closer look at how features are encoded in pre-trained representations. We find that learned representations in a given layer exhibit a degree of diffuse redundancy. Our findings shed light on the nature of representations learned by pre-trained deep neural networks.
  arXiv  Detail & Related papers  (2023-05-31T21:00:50Z)
• Human activity recognition using deep learning approaches and single frame cnn and convolutional lstm [0.0]
  We explore two deep learning-based approaches, namely single frame Convolutional Neural Networks (CNNs) and convolutional Long Short-Term Memory to recognise human actions from videos. The two models were trained and evaluated on a benchmark action recognition dataset, UCF50, and another dataset that was created for the experimentation. Though both models exhibit good accuracies, the single frame CNN model outperforms the Convolutional LSTM model by having an accuracy of 99.8% with the UCF50 dataset.
  arXiv  Detail & Related papers  (2023-04-18T01:33:29Z)
• Mitigating Representation Bias in Action Recognition: Algorithms and Benchmarks [76.35271072704384]
  Deep learning models perform poorly when applied to videos with rare scenes or objects. We tackle this problem from two different angles: algorithm and dataset. We show that the debiased representation can generalize better when transferred to other datasets and tasks.
  arXiv  Detail & Related papers  (2022-09-20T00:30:35Z)
• CONVIQT: Contrastive Video Quality Estimator [63.749184706461826]
  Perceptual video quality assessment (VQA) is an integral component of many streaming and video sharing platforms. Here we consider the problem of learning perceptually relevant video quality representations in a self-supervised manner. Our results indicate that compelling representations with perceptual bearing can be obtained using self-supervised learning.
  arXiv  Detail & Related papers  (2022-06-29T15:22:01Z)
• Backbones-Review: Feature Extraction Networks for Deep Learning and Deep Reinforcement Learning Approaches [3.255610188565679]
  CNNs allow to work on large-scale size of data, as well as cover different scenarios for a specific task. Many networks have been proposed and become the famous networks used for any DL models in any AI task. A backbone is a known network trained in many other tasks before and demonstrates its effectiveness.
  arXiv  Detail & Related papers  (2022-06-16T09:18:34Z)
• Lost Vibration Test Data Recovery Using Convolutional Neural Network: A Case Study [0.0]
  This paper proposes a CNN algorithm for the Alamosa Canyon Bridge as a real structure. Three different CNN models were considered to predict one and two malfunctioned sensors. The accuracy of the model was increased by adding a convolutional layer.
  arXiv  Detail & Related papers  (2022-04-11T23:24:03Z)
• Benchmarking CNN on 3D Anatomical Brain MRI: Architectures, Data Augmentation and Deep Ensemble Learning [2.1446056201053185]
  We propose an extensive benchmark of recent state-of-the-art (SOTA) 3D CNN, evaluating also the benefits of data augmentation and deep ensemble learning. Experiments were conducted on a large multi-site 3D brain anatomical MRI data-set comprising N=10k scans on 3 challenging tasks: age prediction, sex classification, and schizophrenia diagnosis. We found that all models provide significantly better predictions with VBM images than quasi-raw data. DenseNet and tiny-DenseNet, a lighter version that we proposed, provide a good compromise in terms of performance in all data regime
  arXiv  Detail & Related papers  (2021-06-02T13:00:35Z)
• Category-Learning with Context-Augmented Autoencoder [63.05016513788047]
  Finding an interpretable non-redundant representation of real-world data is one of the key problems in Machine Learning. We propose a novel method of using data augmentations when training autoencoders. We train a Variational Autoencoder in such a way, that it makes transformation outcome predictable by auxiliary network.
  arXiv  Detail & Related papers  (2020-10-10T14:04:44Z)
• Curriculum By Smoothing [52.08553521577014]
  Convolutional Neural Networks (CNNs) have shown impressive performance in computer vision tasks such as image classification, detection, and segmentation. We propose an elegant curriculum based scheme that smoothes the feature embedding of a CNN using anti-aliasing or low-pass filters. As the amount of information in the feature maps increases during training, the network is able to progressively learn better representations of the data.
  arXiv  Detail & Related papers  (2020-03-03T07:27:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.