Transformer based neural networks for emotion recognition in conversations

• URL: http://arxiv.org/abs/2405.11222v1
• Date: Sat, 18 May 2024 08:05:05 GMT
• Title: Transformer based neural networks for emotion recognition in conversations
• Authors: Claudiu Creanga, Liviu P. Dinu,
• Abstract summary: Paper outlines approach of the ISDS-NLP team in SemEval 2024 Task 10: Emotion Discovery and Reasoning its Flip in Conversation (EDiReF).
• Score: 4.915541242112533
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper outlines the approach of the ISDS-NLP team in the SemEval 2024 Task 10: Emotion Discovery and Reasoning its Flip in Conversation (EDiReF). For Subtask 1 we obtained a weighted F1 score of 0.43 and placed 12 in the leaderboard. We investigate two distinct approaches: Masked Language Modeling (MLM) and Causal Language Modeling (CLM). For MLM, we employ pre-trained BERT-like models in a multilingual setting, fine-tuning them with a classifier to predict emotions. Experiments with varying input lengths, classifier architectures, and fine-tuning strategies demonstrate the effectiveness of this approach. Additionally, we utilize Mistral 7B Instruct V0.2, a state-of-the-art model, applying zero-shot and few-shot prompting techniques. Our findings indicate that while Mistral shows promise, MLMs currently outperform them in sentence-level emotion classification.

Related papers

• Beyond Silent Letters: Amplifying LLMs in Emotion Recognition with Vocal Nuances [3.396456345114466]
  We propose SpeechCueLLM, a method that translates speech characteristics into natural language descriptions. We evaluate SpeechCueLLM on two datasets: IEMOCAP and MELD, showing significant improvements in emotion recognition accuracy.
  arXiv  Detail & Related papers  (2024-07-31T03:53:14Z)
• Show, Don't Tell: Aligning Language Models with Demonstrated Feedback [54.10302745921713]
  Demonstration ITerated Task Optimization (DITTO) directly aligns language model outputs to a user's demonstrated behaviors. We evaluate DITTO's ability to learn fine-grained style and task alignment across domains such as news articles, emails, and blog posts.
  arXiv  Detail & Related papers  (2024-06-02T23:13:56Z)
• TEII: Think, Explain, Interact and Iterate with Large Language Models to Solve Cross-lingual Emotion Detection [5.942385193284472]
  Cross-lingual emotion detection allows us to analyze global trends, public opinion, and social phenomena at scale. Our system outperformed the baseline by more than 0.16 F1-score absolute, and ranked second amongst competing systems.
  arXiv  Detail & Related papers  (2024-05-27T12:47:40Z)
• Retrofitting Light-weight Language Models for Emotions using Supervised Contrastive Learning [44.17782674872344]
  We present a novel method to induce emotion aspects into pre-trained language models (PLMs) such as BERT and RoBERTa. Our method updates pre-trained network weights using contrastive learning so that the text fragments exhibiting similar emotions are encoded nearby in the representation space.
  arXiv  Detail & Related papers  (2023-10-29T07:43:34Z)
• Bag of Tricks for Effective Language Model Pretraining and Downstream Adaptation: A Case Study on GLUE [93.98660272309974]
  This report briefly describes our submission Vega v1 on the General Language Understanding Evaluation leaderboard. GLUE is a collection of nine natural language understanding tasks, including question answering, linguistic acceptability, sentiment analysis, text similarity, paraphrase detection, and natural language inference. With our optimized pretraining and fine-tuning strategies, our 1.3 billion model sets new state-of-the-art on 4/9 tasks, achieving the best average score of 91.3.
  arXiv  Detail & Related papers  (2023-02-18T09:26:35Z)
• Adapted Multimodal BERT with Layer-wise Fusion for Sentiment Analysis [84.12658971655253]
  We propose Adapted Multimodal BERT, a BERT-based architecture for multimodal tasks. adapter adjusts the pretrained language model for the task at hand, while the fusion layers perform task-specific, layer-wise fusion of audio-visual information with textual BERT representations. In our ablations we see that this approach leads to efficient models, that can outperform their fine-tuned counterparts and are robust to input noise.
  arXiv  Detail & Related papers  (2022-12-01T17:31:42Z)
• Revisiting Self-Training for Few-Shot Learning of Language Model [61.173976954360334]
  Unlabeled data carry rich task-relevant information, they are proven useful for few-shot learning of language model. In this work, we revisit the self-training technique for language model fine-tuning and present a state-of-the-art prompt-based few-shot learner, SFLM.
  arXiv  Detail & Related papers  (2021-10-04T08:51:36Z)
• Frustratingly Simple Pretraining Alternatives to Masked Language Modeling [10.732163031244651]
  Masked language modeling (MLM) is widely used in natural language processing for learning text representations. In this paper, we explore five simple pretraining objectives based on token-level classification tasks as replacements of representations.
  arXiv  Detail & Related papers  (2021-09-04T08:52:37Z)
• W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training [49.47516627019855]
  w2v-BERT is a framework that combines contrastive learning and pre-supervised speech learning. Our experiments show that w2v-BERT achieves competitive results compared to current state-of-the-art pre-trained models.
  arXiv  Detail & Related papers  (2021-08-07T06:29:36Z)
• Explicit Alignment Objectives for Multilingual Bidirectional Encoders [111.65322283420805]
  We present a new method for learning multilingual encoders, AMBER (Aligned Multilingual Bi-directional EncodeR) AMBER is trained on additional parallel data using two explicit alignment objectives that align the multilingual representations at different granularities. Experimental results show that AMBER obtains gains of up to 1.1 average F1 score on sequence tagging and up to 27.3 average accuracy on retrieval over the XLMR-large model.
  arXiv  Detail & Related papers  (2020-10-15T18:34:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.