English Prompts are Better for NLI-based Zero-Shot Emotion Classification than Target-Language Prompts

• URL: http://arxiv.org/abs/2402.03223v4
• Date: Thu, 7 Mar 2024 17:18:18 GMT
• Title: English Prompts are Better for NLI-based Zero-Shot Emotion Classification than Target-Language Prompts
• Authors: Patrick Barei{\ss} and Roman Klinger and Jeremy Barnes
• Abstract summary: We show that it is consistently better to use English prompts even if the data is in a different language. Our experiments with natural language inference-based language models show that it is consistently better to use English prompts even if the data is in a different language.
• Score: 17.099269597133265
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Emotion classification in text is a challenging task due to the processes involved when interpreting a textual description of a potential emotion stimulus. In addition, the set of emotion categories is highly domain-specific. For instance, literature analysis might require the use of aesthetic emotions (e.g., finding something beautiful), and social media analysis could benefit from fine-grained sets (e.g., separating anger from annoyance) than only those that represent basic categories as they have been proposed by Paul Ekman (anger, disgust, fear, joy, surprise, sadness). This renders the task an interesting field for zero-shot classifications, in which the label set is not known at model development time. Unfortunately, most resources for emotion analysis are English, and therefore, most studies on emotion analysis have been performed in English, including those that involve prompting language models for text labels. This leaves us with a research gap that we address in this paper: In which language should we prompt for emotion labels on non-English texts? This is particularly of interest when we have access to a multilingual large language model, because we could request labels with English prompts even for non-English data. Our experiments with natural language inference-based language models show that it is consistently better to use English prompts even if the data is in a different language.

Related papers

• MASIVE: Open-Ended Affective State Identification in English and Spanish [10.41502827362741]
  In this work, we broaden our scope to a practically unbounded set of textitaffective states, which includes any terms that humans use to describe their experiences of feeling. We collect and publish MASIVE, a dataset of Reddit posts in English and Spanish containing over 1,000 unique affective states each. On this task, we find that smaller finetuned multilingual models outperform much larger LLMs, even on region-specific Spanish affective states.
  arXiv  Detail & Related papers  (2024-07-16T21:43:47Z)
• Sociolinguistically Informed Interpretability: A Case Study on Hinglish Emotion Classification [8.010713141364752]
  We study the effect of language on emotion prediction across 3 PLMs on a Hinglish emotion classification dataset. We find that models do learn these associations between language choice and emotional expression. Having code-mixed data present in the pre-training can augment that learning when task-specific data is scarce.
  arXiv  Detail & Related papers  (2024-02-05T16:05:32Z)
• Emotion Rendering for Conversational Speech Synthesis with Heterogeneous Graph-Based Context Modeling [50.99252242917458]
  Conversational Speech Synthesis (CSS) aims to accurately express an utterance with the appropriate prosody and emotional inflection within a conversational setting. To address the issue of data scarcity, we meticulously create emotional labels in terms of category and intensity. Our model outperforms the baseline models in understanding and rendering emotions.
  arXiv  Detail & Related papers  (2023-12-19T08:47:50Z)
• Context Unlocks Emotions: Text-based Emotion Classification Dataset Auditing with Large Language Models [23.670143829183104]
  The lack of contextual information in text data can make the annotation process of text-based emotion classification datasets challenging. We propose a formal definition of textual context to motivate a prompting strategy to enhance such contextual information. Our method improves alignment between inputs and their human-annotated labels from both an empirical and human-evaluated standpoint.
  arXiv  Detail & Related papers  (2023-11-06T21:34:49Z)
• Chat-Capsule: A Hierarchical Capsule for Dialog-level Emotion Analysis [70.98130990040228]
  We propose a Context-based Hierarchical Attention Capsule(Chat-Capsule) model, which models both utterance-level and dialog-level emotions and their interrelations. On a dialog dataset collected from customer support of an e-commerce platform, our model is also able to predict user satisfaction and emotion curve category.
  arXiv  Detail & Related papers  (2022-03-23T08:04:30Z)
• AM2iCo: Evaluating Word Meaning in Context across Low-ResourceLanguages with Adversarial Examples [51.048234591165155]
  We present AM2iCo, Adversarial and Multilingual Meaning in Context. It aims to faithfully assess the ability of state-of-the-art (SotA) representation models to understand the identity of word meaning in cross-lingual contexts. Results reveal that current SotA pretrained encoders substantially lag behind human performance.
  arXiv  Detail & Related papers  (2021-04-17T20:23:45Z)
• SpanEmo: Casting Multi-label Emotion Classification as Span-prediction [15.41237087996244]
  We propose a new model "SpanEmo" casting multi-label emotion classification as span-prediction. We introduce a loss function focused on modelling multiple co-existing emotions in the input sentence. Experiments performed on the SemEval2018 multi-label emotion data over three language sets demonstrate our method's effectiveness.
  arXiv  Detail & Related papers  (2021-01-25T12:11:04Z)
• Leveraging Adversarial Training in Self-Learning for Cross-Lingual Text Classification [52.69730591919885]
  We present a semi-supervised adversarial training process that minimizes the maximal loss for label-preserving input perturbations. We observe significant gains in effectiveness on document and intent classification for a diverse set of languages.
  arXiv  Detail & Related papers  (2020-07-29T19:38:35Z)
• Probing Contextual Language Models for Common Ground with Visual Representations [76.05769268286038]
  We design a probing model that evaluates how effective are text-only representations in distinguishing between matching and non-matching visual representations. Our findings show that language representations alone provide a strong signal for retrieving image patches from the correct object categories. Visually grounded language models slightly outperform text-only language models in instance retrieval, but greatly under-perform humans.
  arXiv  Detail & Related papers  (2020-05-01T21:28:28Z)
• PO-EMO: Conceptualization, Annotation, and Modeling of Aesthetic Emotions in German and English Poetry [26.172030802168752]
  We consider emotions in poetry as they are elicited in the reader, rather than what is expressed in the text or intended by the author. We conceptualize a set of aesthetic emotions that are predictive of aesthetic appreciation in the reader, and allow the annotation of multiple labels per line to capture mixed emotions within their context.
  arXiv  Detail & Related papers  (2020-03-17T13:54:48Z)
• A Deep Neural Framework for Contextual Affect Detection [51.378225388679425]
  A short and simple text carrying no emotion can represent some strong emotions when reading along with its context. We propose a Contextual Affect Detection framework which learns the inter-dependence of words in a sentence.
  arXiv  Detail & Related papers  (2020-01-28T05:03:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.