Reevaluating Data Partitioning for Emotion Detection in EmoWOZ

• URL: http://arxiv.org/abs/2303.13364v1
• Date: Wed, 15 Mar 2023 03:06:13 GMT
• Title: Reevaluating Data Partitioning for Emotion Detection in EmoWOZ
• Authors: Moeen Mostafavi, Michael D. Porter
• Abstract summary: EmoWoz is an extension of MultiWOZ that provides emotion labels for the dialogues. MultiWOZ was partitioned initially for another purpose, resulting in a distributional shift when considering the new purpose of emotion recognition. We propose a stratified sampling scheme based on emotion tags to address this issue, improve the dataset's distribution, and reduce dataset shift.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper focuses on the EmoWoz dataset, an extension of MultiWOZ that provides emotion labels for the dialogues. MultiWOZ was partitioned initially for another purpose, resulting in a distributional shift when considering the new purpose of emotion recognition. The emotion tags in EmoWoz are highly imbalanced and unevenly distributed across the partitions, which causes sub-optimal performance and poor comparison of models. We propose a stratified sampling scheme based on emotion tags to address this issue, improve the dataset's distribution, and reduce dataset shift. We also introduce a special technique to handle conversation (sequential) data with many emotional tags. Using our proposed sampling method, models built upon EmoWoz can perform better, making it a more reliable resource for training conversational agents with emotional intelligence. We recommend that future researchers use this new partitioning to ensure consistent and accurate performance evaluations.

Related papers

• Emotion Detection in Reddit: Comparative Study of Machine Learning and Deep Learning Techniques [0.0]
  This study concentrates on text-based emotion detection by leveraging the GoEmotions dataset. We employed a range of models for this task, including six machine learning models, three ensemble models, and a Long Short-Term Memory (LSTM) model. Results indicate that the Stacking classifier outperforms other models in accuracy and performance.
  arXiv  Detail & Related papers  (2024-11-15T16:28:25Z)
• Towards Empathetic Conversational Recommender Systems [77.53167131692]
  We propose an empathetic conversational recommender (ECR) framework. ECR contains two main modules: emotion-aware item recommendation and emotion-aligned response generation. Our experiments on the ReDial dataset validate the efficacy of our framework in enhancing recommendation accuracy and improving user satisfaction.
  arXiv  Detail & Related papers  (2024-08-30T15:43:07Z)
• The Whole Is Bigger Than the Sum of Its Parts: Modeling Individual Annotators to Capture Emotional Variability [7.1394038985662664]
  Emotion expression and perception are nuanced, complex, and highly subjective processes. Most speech emotion recognition tasks address this by averaging annotator labels as ground truth. Previous work has attempted to learn distributions to capture emotion variability, but these methods also lose information about the individual annotators. We introduce a novel method to create distributions from continuous model outputs that permit the learning of emotion distributions during model training.
  arXiv  Detail & Related papers  (2024-08-21T19:24:06Z)
• Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning [55.127202990679976]
  We introduce the MERR dataset, containing 28,618 coarse-grained and 4,487 fine-grained annotated samples across diverse emotional categories. This dataset enables models to learn from varied scenarios and generalize to real-world applications. We propose Emotion-LLaMA, a model that seamlessly integrates audio, visual, and textual inputs through emotion-specific encoders.
  arXiv  Detail & Related papers  (2024-06-17T03:01:22Z)
• Emotion-cause pair extraction method based on multi-granularity information and multi-module interaction [0.6577148087211809]
  The purpose of emotion-cause pair extraction is to extract the pair of emotion clauses and cause clauses. Existing models do not adequately address the emotion and cause-induced locational imbalance of samples. We propose an end-to-end multitasking model (MM-ECPE) based on shared interaction between GRU, knowledge graph and transformer modules.
  arXiv  Detail & Related papers  (2024-04-10T08:00:26Z)
• Emotion Rendering for Conversational Speech Synthesis with Heterogeneous Graph-Based Context Modeling [50.99252242917458]
  Conversational Speech Synthesis (CSS) aims to accurately express an utterance with the appropriate prosody and emotional inflection within a conversational setting. To address the issue of data scarcity, we meticulously create emotional labels in terms of category and intensity. Our model outperforms the baseline models in understanding and rendering emotions.
  arXiv  Detail & Related papers  (2023-12-19T08:47:50Z)
• Language Models (Mostly) Do Not Consider Emotion Triggers When Predicting Emotion [87.18073195745914]
  We investigate how well human-annotated emotion triggers correlate with features deemed salient in their prediction of emotions. Using EmoTrigger, we evaluate the ability of large language models to identify emotion triggers. Our analysis reveals that emotion triggers are largely not considered salient features for emotion prediction models, instead there is intricate interplay between various features and the task of emotion detection.
  arXiv  Detail & Related papers  (2023-11-16T06:20:13Z)
• Dynamic Causal Disentanglement Model for Dialogue Emotion Detection [77.96255121683011]
  We propose a Dynamic Causal Disentanglement Model based on hidden variable separation. This model effectively decomposes the content of dialogues and investigates the temporal accumulation of emotions. Specifically, we propose a dynamic temporal disentanglement model to infer the propagation of utterances and hidden variables.
  arXiv  Detail & Related papers  (2023-09-13T12:58:09Z)
• Automatic Emotion Modelling in Written Stories [4.484753247472559]
  We propose a set of novel Transformer-based methods for predicting emotional signals over the course of written stories. We explore several strategies for fine-tuning a pretrained ELECTRA model and study the benefits of considering a sentence's context. Our code and additional annotations are made available at https://github.com/lc0197/emotion_modelling_stories.
  arXiv  Detail & Related papers  (2022-12-21T21:46:01Z)
• Persian Emotion Detection using ParsBERT and Imbalanced Data Handling Approaches [0.0]
  EmoPars and ArmanEmo are two new human-labeled emotion datasets for the Persian language. We evaluate EmoPars and compare them with ArmanEmo. Our model reaches a Macro-averaged F1-score of 0.81 and 0.76 on ArmanEmo and EmoPars, respectively.
  arXiv  Detail & Related papers  (2022-11-15T10:22:49Z)
• Multimodal Emotion Recognition using Transfer Learning from Speaker Recognition and BERT-based models [53.31917090073727]
  We propose a neural network-based emotion recognition framework that uses a late fusion of transfer-learned and fine-tuned models from speech and text modalities. We evaluate the effectiveness of our proposed multimodal approach on the interactive emotional dyadic motion capture dataset.
  arXiv  Detail & Related papers  (2022-02-16T00:23:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.