Robust Emotion Recognition in Context Debiasing

• URL: http://arxiv.org/abs/2403.05963v3
• Date: Sun, 2 Jun 2024 01:35:28 GMT
• Title: Robust Emotion Recognition in Context Debiasing
• Authors: Dingkang Yang, Kun Yang, Mingcheng Li, Shunli Wang, Shuaibing Wang, Lihua Zhang,
• Abstract summary: Context-aware emotion recognition (CAER) has recently boosted the practical applications of affective computing techniques in unconstrained environments. Despite advancements, the biggest challenge remains due to context bias interference. We propose a counterfactual emotion inference (CLEF) framework to address the above issue.
• Score: 12.487614699507793
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Context-aware emotion recognition (CAER) has recently boosted the practical applications of affective computing techniques in unconstrained environments. Mainstream CAER methods invariably extract ensemble representations from diverse contexts and subject-centred characteristics to perceive the target person's emotional state. Despite advancements, the biggest challenge remains due to context bias interference. The harmful bias forces the models to rely on spurious correlations between background contexts and emotion labels in likelihood estimation, causing severe performance bottlenecks and confounding valuable context priors. In this paper, we propose a counterfactual emotion inference (CLEF) framework to address the above issue. Specifically, we first formulate a generalized causal graph to decouple the causal relationships among the variables in CAER. Following the causal graph, CLEF introduces a non-invasive context branch to capture the adverse direct effect caused by the context bias. During the inference, we eliminate the direct context effect from the total causal effect by comparing factual and counterfactual outcomes, resulting in bias mitigation and robust prediction. As a model-agnostic framework, CLEF can be readily integrated into existing methods, bringing consistent performance gains.

Related papers

• Towards Context-Aware Emotion Recognition Debiasing from a Causal Demystification Perspective via De-confounded Training [14.450673163785094]
  Context-Aware Emotion Recognition (CAER) provides valuable semantic cues for recognizing the emotions of target persons. Current approaches invariably focus on designing sophisticated structures to extract perceptually critical representations from contexts. We present a Contextual Causal Intervention Module (CCIM) to de-confound the confounder.
  arXiv  Detail & Related papers  (2024-07-06T05:29:02Z)
• Interventional Fairness on Partially Known Causal Graphs: A Constrained Optimization Approach [44.48385991344273]
  We propose a framework for achieving causal fairness based on the notion of interventions when the true causal graph is partially known. The proposed approach involves modeling fair prediction using a class of causal DAGs that can be learned from observational data combined with domain knowledge. Results on both simulated and real-world datasets demonstrate the effectiveness of this method.
  arXiv  Detail & Related papers  (2024-01-19T11:20:31Z)
• Context De-confounded Emotion Recognition [12.037240778629346]
  Context-Aware Emotion Recognition (CAER) aims to perceive the emotional states of the target person with contextual information. A long-overlooked issue is that a context bias in existing datasets leads to a significantly unbalanced distribution of emotional states. This paper provides a causality-based perspective to disentangle the models from the impact of such bias, and formulate the causalities among variables in the CAER task.
  arXiv  Detail & Related papers  (2023-03-21T15:12:20Z)
• The Invariant Ground Truth of Affect [2.570570340104555]
  Ground truth of affect is attributed to the affect labels which inadvertently include biases inherent to the subjective nature of emotion and its labeling. This paper reframes the ways one may obtain a reliable ground truth of affect by transferring aspects of causation theory to affective computing. We employ causation inspired methods for detecting outliers in affective corpora and building affect models that are robust across participants and tasks.
  arXiv  Detail & Related papers  (2022-10-14T08:26:01Z)
• An Ordinal Latent Variable Model of Conflict Intensity [59.49424978353101]
  The Goldstein scale is a widely-used expert-based measure that scores events on a conflictual-cooperative scale. This paper takes a latent variable-based approach to measuring conflict intensity.
  arXiv  Detail & Related papers  (2022-10-08T08:59:17Z)
• Counterfactual Reasoning for Out-of-distribution Multimodal Sentiment Analysis [56.84237932819403]
  This paper aims to estimate and mitigate the bad effect of textual modality for strong OOD generalization. Inspired by this, we devise a model-agnostic counterfactual framework for multimodal sentiment analysis.
  arXiv  Detail & Related papers  (2022-07-24T03:57:40Z)
• Causal Transportability for Visual Recognition [70.13627281087325]
  We show that standard classifiers fail because the association between images and labels is not transportable across settings. We then show that the causal effect, which severs all sources of confounding, remains invariant across domains. This motivates us to develop an algorithm to estimate the causal effect for image classification.
  arXiv  Detail & Related papers  (2022-04-26T15:02:11Z)
• Towards Unbiased Visual Emotion Recognition via Causal Intervention [63.74095927462]
  We propose a novel Emotion Recognition Network (IERN) to alleviate the negative effects brought by the dataset bias. A series of designed tests validate the effectiveness of IERN, and experiments on three emotion benchmarks demonstrate that IERN outperforms other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-07-26T10:40:59Z)
• Adversarial Robustness through the Lens of Causality [105.51753064807014]
  adversarial vulnerability of deep neural networks has attracted significant attention in machine learning. We propose to incorporate causality into mitigating adversarial vulnerability. Our method can be seen as the first attempt to leverage causality for mitigating adversarial vulnerability.
  arXiv  Detail & Related papers  (2021-06-11T06:55:02Z)
• Learning Causal Semantic Representation for Out-of-Distribution Prediction [125.38836464226092]
  We propose a Causal Semantic Generative model (CSG) based on a causal reasoning so that the two factors are modeled separately. We show that CSG can identify the semantic factor by fitting training data, and this semantic-identification guarantees the boundedness of OOD generalization error.
  arXiv  Detail & Related papers  (2020-11-03T13:16:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.