Navigating Nuance: In Quest for Political Truth

• URL: http://arxiv.org/abs/2501.00782v1
• Date: Wed, 01 Jan 2025 09:24:47 GMT
• Title: Navigating Nuance: In Quest for Political Truth
• Authors: Soumyadeep Sar, Dwaipayan Roy,
• Abstract summary: We evaluate the performance of the Llama-3 (70B) language model on the Media Bias Identification Benchmark (MBIB) Our findings underscore the challenges of detecting political bias and highlight the potential of transfer learning methods to enhance future models.
• Score: 1.4127714091330967
• License:
• Abstract: This study investigates the several nuanced rationales for countering the rise of political bias. We evaluate the performance of the Llama-3 (70B) language model on the Media Bias Identification Benchmark (MBIB), based on a novel prompting technique that incorporates subtle reasons for identifying political leaning. Our findings underscore the challenges of detecting political bias and highlight the potential of transfer learning methods to enhance future models. Through our framework, we achieve a comparable performance with the supervised and fully fine-tuned ConvBERT model, which is the state-of-the-art model, performing best among other baseline models for the political bias task on MBIB. By demonstrating the effectiveness of our approach, we contribute to the development of more robust tools for mitigating the spread of misinformation and polarization. Our codes and dataset are made publicly available in github.

Related papers

• Balancing Transparency and Accuracy: A Comparative Analysis of Rule-Based and Deep Learning Models in Political Bias Classification [5.550237524713089]
  The study highlights the sensitivity of modern self-learning systems to unconstrained data ingestion. Applying both models to left-leaning (CNN) and right-leaning (FOX) news articles, we assess their effectiveness on data beyond the original training and test sets. We contrast the opaque architecture of a deep learning model with the transparency of a linguistically informed rule-based model.
  arXiv  Detail & Related papers  (2024-11-07T00:09:18Z)
• Thinking Racial Bias in Fair Forgery Detection: Models, Datasets and Evaluations [63.52709761339949]
  We first contribute a dedicated dataset called the Fair Forgery Detection (FairFD) dataset, where we prove the racial bias of public state-of-the-art (SOTA) methods. We design novel metrics including Approach Averaged Metric and Utility Regularized Metric, which can avoid deceptive results. We also present an effective and robust post-processing technique, Bias Pruning with Fair Activations (BPFA), which improves fairness without requiring retraining or weight updates.
  arXiv  Detail & Related papers  (2024-07-19T14:53:18Z)
• Representation Bias in Political Sample Simulations with Large Language Models [54.48283690603358]
  This study seeks to identify and quantify biases in simulating political samples with Large Language Models. Using the GPT-3.5-Turbo model, we leverage data from the American National Election Studies, German Longitudinal Election Study, Zuobiao dataset, and China Family Panel Studies.
  arXiv  Detail & Related papers  (2024-07-16T05:52:26Z)
• P^3SUM: Preserving Author's Perspective in News Summarization with Diffusion Language Models [57.571395694391654]
  We find that existing approaches alter the political opinions and stances of news articles in more than 50% of summaries. We propose P3SUM, a diffusion model-based summarization approach controlled by political perspective classifiers. Experiments on three news summarization datasets demonstrate that P3SUM outperforms state-of-the-art summarization systems.
  arXiv  Detail & Related papers  (2023-11-16T10:14:28Z)
• Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis [86.49858739347412]
  Large Language Models (LLMs) have sparked intense debate regarding the prevalence of bias in these models and its mitigation. We propose a prompt-based method for the extraction of confounding and mediating attributes which contribute to the decision process. We find that the observed disparate treatment can at least in part be attributed to confounding and mitigating attributes and model misalignment.
  arXiv  Detail & Related papers  (2023-11-15T00:02:25Z)
• Quantitative Analysis of Forecasting Models:In the Aspect of Online Political Bias [0.0]
  We propose a approach to classify social media posts into five distinct political leaning categories. Our approach involves utilizing existing time series forecasting models on two social media datasets with different political ideologies.
  arXiv  Detail & Related papers  (2023-09-11T16:17:24Z)
• Mitigating Political Bias in Language Models Through Reinforced Calibration [6.964628305312507]
  We describe metrics for measuring political bias in GPT-2 generation. We propose a reinforcement learning (RL) framework for mitigating political biases in generated text.
  arXiv  Detail & Related papers  (2021-04-30T07:21:30Z)
• Delving into Data: Effectively Substitute Training for Black-box Attack [84.85798059317963]
  We propose a novel perspective substitute training that focuses on designing the distribution of data used in the knowledge stealing process. The combination of these two modules can further boost the consistency of the substitute model and target model, which greatly improves the effectiveness of adversarial attack.
  arXiv  Detail & Related papers  (2021-04-26T07:26:29Z)
• Inflating Topic Relevance with Ideology: A Case Study of Political Ideology Bias in Social Topic Detection Models [16.279854003220418]
  We investigate the impact of political ideology biases in training data. Our work highlights the susceptibility of large, complex models to propagating the biases from human-selected input. As a way to mitigate the bias, we propose to learn a text representation that is invariant to political ideology while still judging topic relevance.
  arXiv  Detail & Related papers  (2020-11-29T05:54:03Z)
• We Can Detect Your Bias: Predicting the Political Ideology of News Articles [35.761722515882646]
  We release a dataset of 34,737 articles that were manually annotated for political ideology. We use a challenging experimental setup where the test examples come from media that were not seen during training. Our experimental results show very sizable improvements over using state-of-the-art pre-trained Transformers.
  arXiv  Detail & Related papers  (2020-10-11T20:27:55Z)
• On the model-based stochastic value gradient for continuous reinforcement learning [50.085645237597056]
  We show that simple model-based agents can outperform state-of-the-art model-free agents in terms of both sample-efficiency and final reward. Our findings suggest that model-based policy evaluation deserves closer attention.
  arXiv  Detail & Related papers  (2020-08-28T17:58:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.