Affective Processes: stochastic modelling of temporal context for emotion and facial expression recognition

• URL: http://arxiv.org/abs/2103.13372v1
• Date: Wed, 24 Mar 2021 17:48:19 GMT
• Title: Affective Processes: stochastic modelling of temporal context for emotion and facial expression recognition
• Authors: Enrique Sanchez and Mani Kumar Tellamekala and Michel Valstar and Georgios Tzimiropoulos
• Abstract summary: We build upon the framework of Neural Processes to propose a method for apparent emotion recognition with three key components. We validate our approach on four databases, two for Valence and Arousal estimation and two for Action Unit intensity estimation. Results show a consistent improvement over a series of strong baselines as well as over state-of-the-art methods.
• Score: 38.47712256338113
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Temporal context is key to the recognition of expressions of emotion. Existing methods, that rely on recurrent or self-attention models to enforce temporal consistency, work on the feature level, ignoring the task-specific temporal dependencies, and fail to model context uncertainty. To alleviate these issues, we build upon the framework of Neural Processes to propose a method for apparent emotion recognition with three key novel components: (a) probabilistic contextual representation with a global latent variable model; (b) temporal context modelling using task-specific predictions in addition to features; and (c) smart temporal context selection. We validate our approach on four databases, two for Valence and Arousal estimation (SEWA and AffWild2), and two for Action Unit intensity estimation (DISFA and BP4D). Results show a consistent improvement over a series of strong baselines as well as over state-of-the-art methods.

Related papers

• Hierarchical Relation-augmented Representation Generalization for Few-shot Action Recognition [53.02634128715853]
  Few-shot action recognition (FSAR) aims to recognize novel action categories with few exemplars. We propose HR2G-shot, a Hierarchical Relation-augmented Representation Generalization framework for FSAR. It unifies three types of relation modeling (inter-frame, inter-video, and inter-task) to learn task-specific temporal patterns from a holistic view.
  arXiv  Detail & Related papers  (2025-04-14T10:23:22Z)
• Robust Dynamic Facial Expression Recognition [6.626374248579249]
  This paper proposes a robust method of distinguishing between hard and noisy samples. To identify the principal expression in a video, a key expression re-sampling framework and a dual-stream hierarchical network is proposed. The proposed method has been shown to outperform current State-Of-The-Art approaches in DFER.
  arXiv  Detail & Related papers  (2025-02-22T07:48:12Z)
• On the Identification of Temporally Causal Representation with Instantaneous Dependence [50.14432597910128]
  Temporally causal representation learning aims to identify the latent causal process from time series observations. Most methods require the assumption that the latent causal processes do not have instantaneous relations. We propose an textbfIDentification framework for instantanetextbfOus textbfLatent dynamics.
  arXiv  Detail & Related papers  (2024-05-24T08:08:05Z)
• Relational Temporal Graph Reasoning for Dual-task Dialogue Language Understanding [39.76268402567324]
  Dual-task dialog understanding language aims to tackle two correlative dialog language understanding tasks simultaneously via their inherent correlations. We put forward a new framework, whose core is relational temporal graph reasoning. Our models outperform state-of-the-art models by a large margin.
  arXiv  Detail & Related papers  (2023-06-15T13:19:08Z)
• Spatio-temporal Relation Modeling for Few-shot Action Recognition [100.3999454780478]
  We propose a few-shot action recognition framework, STRM, which enhances class-specific featureriminability while simultaneously learning higher-order temporal representations. Our approach achieves an absolute gain of 3.5% in classification accuracy, as compared to the best existing method in the literature.
  arXiv  Detail & Related papers  (2021-12-09T18:59:14Z)
• An Empirical Study: Extensive Deep Temporal Point Process [23.9359814366167]
  We first review recent research emphasis and difficulties in modeling asynchronous event sequences with deep temporal pointprocess. We propose a Granger causality discovery framework for exploiting the relations among multi-types of events.
  arXiv  Detail & Related papers  (2021-10-19T10:15:00Z)
• Progressive Spatio-Temporal Bilinear Network with Monte Carlo Dropout for Landmark-based Facial Expression Recognition with Uncertainty Estimation [93.73198973454944]
  The performance of our method is evaluated on three widely used datasets. It is comparable to that of video-based state-of-the-art methods while it has much less complexity.
  arXiv  Detail & Related papers  (2021-06-08T13:40:30Z)
• Exploiting Emotional Dependencies with Graph Convolutional Networks for Facial Expression Recognition [31.40575057347465]
  This paper proposes a novel multi-task learning framework to recognize facial expressions in-the-wild. A shared feature representation is learned for both discrete and continuous recognition in a MTL setting. The results of our experiments show that our method outperforms the current state-of-the-art methods on discrete FER.
  arXiv  Detail & Related papers  (2021-06-07T10:20:05Z)
• STAGE: Tool for Automated Extraction of Semantic Time Cues to Enrich Neural Temporal Ordering Models [4.6150532698347835]
  We develop STAGE, a system that can automatically extract time cues and convert them into representations suitable for integration with neural models. We demonstrate promising results on two event ordering datasets, and highlight important issues in semantic cue representation and integration for future research.
  arXiv  Detail & Related papers  (2021-05-15T23:34:02Z)
• A Multi-term and Multi-task Analyzing Framework for Affective Analysis in-the-wild [0.2216657815393579]
  We introduce the affective recognition method that was submitted to the Affective Behavior Analysis in-the-wild (ABAW) 2020 Contest. Since affective behaviors have many observable features that have their own time frames, we introduced multiple optimized time windows. We generated affective recognition models for each time window and ensembled these models together.
  arXiv  Detail & Related papers  (2020-09-29T09:24:29Z)
• Modeling Inter-Aspect Dependencies with a Non-temporal Mechanism for Aspect-Based Sentiment Analysis [70.22725610210811]
  We propose a novel non-temporal mechanism to enhance the ABSA task through modeling inter-aspect dependencies. We focus on the well-known class imbalance issue on the ABSA task and address it by down-weighting the loss assigned to well-classified instances.
  arXiv  Detail & Related papers  (2020-08-12T08:50:09Z)
• Uncertainty Quantification for Deep Context-Aware Mobile Activity Recognition and Unknown Context Discovery [85.36948722680822]
  We develop a context-aware mixture of deep models termed the alpha-beta network. We improve accuracy and F score by 10% by identifying high-level contexts. In order to ensure training stability, we have used a clustering-based pre-training in both public and in-house datasets.
  arXiv  Detail & Related papers  (2020-03-03T19:35:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.