Uncovering Political Bias in Emotion Inference Models: Implications for sentiment analysis in social science research

• URL: http://arxiv.org/abs/2407.13891v1
• Date: Thu, 18 Jul 2024 20:31:07 GMT
• Title: Uncovering Political Bias in Emotion Inference Models: Implications for sentiment analysis in social science research
• Authors: Hubert Plisiecki, Paweł Lenartowicz, Maria Flakus, Artur Pokropek,
• Abstract summary: This paper investigates the presence of political bias in machine learning models used for sentiment analysis (SA) in social science research. We conducted a bias audit on a Polish sentiment analysis model developed in our lab. Our findings indicate that annotations by human raters propagate political biases into the model's predictions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: This paper investigates the presence of political bias in emotion inference models used for sentiment analysis (SA) in social science research. Machine learning models often reflect biases in their training data, impacting the validity of their outcomes. While previous research has highlighted gender and race biases, our study focuses on political bias - an underexplored yet pervasive issue that can skew the interpretation of text data across a wide array of studies. We conducted a bias audit on a Polish sentiment analysis model developed in our lab. By analyzing valence predictions for names and sentences involving Polish politicians, we uncovered systematic differences influenced by political affiliations. Our findings indicate that annotations by human raters propagate political biases into the model's predictions. To mitigate this, we pruned the training dataset of texts mentioning these politicians and observed a reduction in bias, though not its complete elimination. Given the significant implications of political bias in SA, our study emphasizes caution in employing these models for social science research. We recommend a critical examination of SA results and propose using lexicon-based systems as a more ideologically neutral alternative. This paper underscores the necessity for ongoing scrutiny and methodological adjustments to ensure the reliability and impartiality of the use of machine learning in academic and applied contexts.

Related papers

• Representation Bias in Political Sample Simulations with Large Language Models [54.48283690603358]
  This study seeks to identify and quantify biases in simulating political samples with Large Language Models. Using the GPT-3.5-Turbo model, we leverage data from the American National Election Studies, German Longitudinal Election Study, Zuobiao dataset, and China Family Panel Studies.
  arXiv  Detail & Related papers  (2024-07-16T05:52:26Z)
• Inducing Political Bias Allows Language Models Anticipate Partisan Reactions to Controversies [5.958974943807783]
  This study addresses the challenge of understanding political bias in digitized discourse using Large Language Models (LLMs) We present a comprehensive analytical framework, consisting of Partisan Bias Divergence Assessment and Partisan Class Tendency Prediction. Our findings reveal the model's effectiveness in capturing emotional and moral nuances, albeit with some challenges in stance detection.
  arXiv  Detail & Related papers  (2023-11-16T08:57:53Z)
• Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis [86.49858739347412]
  Large Language Models (LLMs) have sparked intense debate regarding the prevalence of bias in these models and its mitigation. We propose a prompt-based method for the extraction of confounding and mediating attributes which contribute to the decision process. We find that the observed disparate treatment can at least in part be attributed to confounding and mitigating attributes and model misalignment.
  arXiv  Detail & Related papers  (2023-11-15T00:02:25Z)
• Quantitative Analysis of Forecasting Models:In the Aspect of Online Political Bias [0.0]
  We propose a approach to classify social media posts into five distinct political leaning categories. Our approach involves utilizing existing time series forecasting models on two social media datasets with different political ideologies.
  arXiv  Detail & Related papers  (2023-09-11T16:17:24Z)
• Unveiling Gender Bias in Terms of Profession Across LLMs: Analyzing and Addressing Sociological Implications [0.0]
  The study examines existing research on gender bias in AI language models and identifies gaps in the current knowledge. The findings shed light on gendered word associations, language usage, and biased narratives present in the outputs of Large Language Models. The paper presents strategies for reducing gender bias in LLMs, including algorithmic approaches and data augmentation techniques.
  arXiv  Detail & Related papers  (2023-07-18T11:38:45Z)
• From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models [40.89699305385269]
  Language models (LMs) are pretrained on diverse data sources, including news, discussion forums, books, and online encyclopedias. We develop new methods to measure political biases in LMs trained on such corpora, along social and economic axes, and on top of politically biased LMs. We focus on hate speech and misinformation detection, aiming to empirically quantify the effects of political (social, economic) biases in pretraining data on the fairness of high-stakes social-oriented tasks.
  arXiv  Detail & Related papers  (2023-05-15T00:06:30Z)
• Anatomizing Bias in Facial Analysis [86.79402670904338]
  Existing facial analysis systems have been shown to yield biased results against certain demographic subgroups. It has become imperative to ensure that these systems do not discriminate based on gender, identity, or skin tone of individuals. This has led to research in the identification and mitigation of bias in AI systems.
  arXiv  Detail & Related papers  (2021-12-13T09:51:13Z)
• Towards Understanding and Mitigating Social Biases in Language Models [107.82654101403264]
  Large-scale pretrained language models (LMs) can be potentially dangerous in manifesting undesirable representational biases. We propose steps towards mitigating social biases during text generation. Our empirical results and human evaluation demonstrate effectiveness in mitigating bias while retaining crucial contextual information.
  arXiv  Detail & Related papers  (2021-06-24T17:52:43Z)
• Inflating Topic Relevance with Ideology: A Case Study of Political Ideology Bias in Social Topic Detection Models [16.279854003220418]
  We investigate the impact of political ideology biases in training data. Our work highlights the susceptibility of large, complex models to propagating the biases from human-selected input. As a way to mitigate the bias, we propose to learn a text representation that is invariant to political ideology while still judging topic relevance.
  arXiv  Detail & Related papers  (2020-11-29T05:54:03Z)
• Towards Debiasing Sentence Representations [109.70181221796469]
  We show that Sent-Debias is effective in removing biases, and at the same time, preserves performance on sentence-level downstream tasks. We hope that our work will inspire future research on characterizing and removing social biases from widely adopted sentence representations for fairer NLP.
  arXiv  Detail & Related papers  (2020-07-16T04:22:30Z)
• Sentiment Analysis Based on Deep Learning: A Comparative Study [69.09570726777817]
  The study of public opinion can provide us with valuable information. The efficiency and accuracy of sentiment analysis is being hindered by the challenges encountered in natural language processing. This paper reviews the latest studies that have employed deep learning to solve sentiment analysis problems.
  arXiv  Detail & Related papers  (2020-06-05T16:28:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.