What is Sentiment Meant to Mean to Language Models?

• URL: http://arxiv.org/abs/2405.02454v1
• Date: Fri, 3 May 2024 19:37:37 GMT
• Title: What is Sentiment Meant to Mean to Language Models?
• Authors: Michael Burnham,
• Abstract summary: "sentiment" entails a wide variety of concepts depending on the domain and tools used. "sentiment" has been used to mean emotion, opinions, market movements, or simply a general good-bad'' dimension.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sentiment analysis is one of the most widely used techniques in text analysis. Recent advancements with Large Language Models have made it more accurate and accessible than ever, allowing researchers to classify text with only a plain English prompt. However, "sentiment" entails a wide variety of concepts depending on the domain and tools used. It has been used to mean emotion, opinions, market movements, or simply a general ``good-bad'' dimension. This raises a question: What exactly are language models doing when prompted to label documents by sentiment? This paper first overviews how sentiment is defined across different contexts, highlighting that it is a confounded measurement construct in that it entails multiple variables, such as emotional valence and opinion, without disentangling them. I then test three language models across two data sets with prompts requesting sentiment, valence, and stance classification. I find that sentiment labels most strongly correlate with valence labels. I further find that classification improves when researchers more precisely specify their dimension of interest rather than using the less well-defined concept of sentiment. I conclude by encouraging researchers to move beyond "sentiment" when feasible and use a more precise measurement construct.

Related papers

• You Shall Know a Tool by the Traces it Leaves: The Predictability of Sentiment Analysis Tools [74.98850427240464]
  We show that sentiment analysis tools disagree on the same dataset. We show that the sentiment tool used for sentiment annotation can even be predicted from its outcome.
  arXiv  Detail & Related papers  (2024-10-18T17:27:38Z)
• Entity-Level Sentiment: More than the Sum of Its Parts [13.829487868948686]
  We show that the reader's perceived sentiment regarding an entity often differs from an arithmetic aggregation of sentiments at the sentence level. Our dataset reveals the complexity of entity-specific sentiment in longer texts.
  arXiv  Detail & Related papers  (2024-07-04T13:21:07Z)
• English Prompts are Better for NLI-based Zero-Shot Emotion Classification than Target-Language Prompts [17.099269597133265]
  We show that it is consistently better to use English prompts even if the data is in a different language. Our experiments with natural language inference-based language models show that it is consistently better to use English prompts even if the data is in a different language.
  arXiv  Detail & Related papers  (2024-02-05T17:36:19Z)
• Linguistic features for sentence difficulty prediction in ABSA [0.3172761915061083]
  We study the impact of domain diversity and syntactic diversity on difficulty. We employ two ways of defining sentence difficulty. We also define 9 linguistic features that, combined, aim at estimating the difficulty at sentence level.
  arXiv  Detail & Related papers  (2024-02-05T16:31:03Z)
• A Soft Contrastive Learning-based Prompt Model for Few-shot Sentiment Analysis [38.17825180485807]
  We propose a Soft Contrastive learning-based Prompt model for few-shot sentiment analysis. First, we design a sentiment-aware chain of thought prompt module to guide the model to predict the sentiment. Then, we propose a soft contrastive learning algorithm to take the correlation of the labels into account.
  arXiv  Detail & Related papers  (2023-12-16T15:17:28Z)
• Syntax and Semantics Meet in the "Middle": Probing the Syntax-Semantics Interface of LMs Through Agentivity [68.8204255655161]
  We present the semantic notion of agentivity as a case study for probing such interactions. This suggests LMs may potentially serve as more useful tools for linguistic annotation, theory testing, and discovery.
  arXiv  Detail & Related papers  (2023-05-29T16:24:01Z)
• Towards Unsupervised Recognition of Token-level Semantic Differences in Related Documents [61.63208012250885]
  We formulate recognizing semantic differences as a token-level regression task. We study three unsupervised approaches that rely on a masked language model. Our results show that an approach based on word alignment and sentence-level contrastive learning has a robust correlation to gold labels.
  arXiv  Detail & Related papers  (2023-05-22T17:58:04Z)
• Sentiment-Aware Word and Sentence Level Pre-training for Sentiment Analysis [64.70116276295609]
  SentiWSP is a Sentiment-aware pre-trained language model with combined Word-level and Sentence-level Pre-training tasks. SentiWSP achieves new state-of-the-art performance on various sentence-level and aspect-level sentiment classification benchmarks.
  arXiv  Detail & Related papers  (2022-10-18T12:25:29Z)
• KESA: A Knowledge Enhanced Approach For Sentiment Analysis [13.937274761540925]
  We study sentence-level sentiment analysis and propose two sentiment-aware auxiliary tasks named sentiment word cloze and conditional sentiment prediction. The experimental results demonstrate that our approach consistently outperforms pre-trained models.
  arXiv  Detail & Related papers  (2022-02-24T13:21:27Z)
• Sentiment analysis in tweets: an assessment study from classical to modern text representation models [59.107260266206445]
  Short texts published on Twitter have earned significant attention as a rich source of information. Their inherent characteristics, such as the informal, and noisy linguistic style, remain challenging to many natural language processing (NLP) tasks. This study fulfils an assessment of existing language models in distinguishing the sentiment expressed in tweets by using a rich collection of 22 datasets.
  arXiv  Detail & Related papers  (2021-05-29T21:05:28Z)
• A computational model implementing subjectivity with the 'Room Theory'. The case of detecting Emotion from Text [68.8204255655161]
  This work introduces a new method to consider subjectivity and general context dependency in text analysis. By using similarity measure between words, we are able to extract the relative relevance of the elements in the benchmark. This method could be applied to all the cases where evaluating subjectivity is relevant to understand the relative value or meaning of a text.
  arXiv  Detail & Related papers  (2020-05-12T21:26:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.