Related papers: Read Between the Lines: A Benchmark for Uncovering Political Bias in Bangla News Articles

Read Between the Lines: A Benchmark for Uncovering Political Bias in Bangla News Articles

URL: http://arxiv.org/abs/2510.03898v1
Date: Sat, 04 Oct 2025 18:34:34 GMT
Title: Read Between the Lines: A Benchmark for Uncovering Political Bias in Bangla News Articles
Authors: Nusrat Jahan Lia, Shubhashis Roy Dipta, Abdullah Khan Zehady, Naymul Islam, Madhusodan Chakraborty, Abdullah Al Wasif,
Abstract summary: Political stance detection in Bangla news requires understanding of linguistic cues, cultural context, subtle biases, rhetorical strategies, code-switching, implicit sentiment, and socio-political background.<n>We introduce the first benchmark dataset of 200 politically significant and highly debated Bangla news articles, labeled for government-leaning, government-critique, and neutral stances, alongside diagnostic analyses for evaluating large language models (LLMs)<n>Our evaluation of 28 proprietary and open-source LLMs shows strong performance in detecting government-critique content (F1 up to 0.83) but substantial difficulty with neutral articles (F1 as low as 0.00)
Score: 0.0
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Detecting media bias is crucial, specifically in the South Asian region. Despite this, annotated datasets and computational studies for Bangla political bias research remain scarce. Crucially because, political stance detection in Bangla news requires understanding of linguistic cues, cultural context, subtle biases, rhetorical strategies, code-switching, implicit sentiment, and socio-political background. To address this, we introduce the first benchmark dataset of 200 politically significant and highly debated Bangla news articles, labeled for government-leaning, government-critique, and neutral stances, alongside diagnostic analyses for evaluating large language models (LLMs). Our comprehensive evaluation of 28 proprietary and open-source LLMs shows strong performance in detecting government-critique content (F1 up to 0.83) but substantial difficulty with neutral articles (F1 as low as 0.00). Models also tend to over-predict government-leaning stances, often misinterpreting ambiguous narratives. This dataset and its associated diagnostics provide a foundation for advancing stance detection in Bangla media research and offer insights for improving LLM performance in low-resource languages.

Related papers

Debiasing Large Language Models in Thai Political Stance Detection via Counterfactual Calibration [0.0]
Thai politics is marked by indirect language, polarized figures, and entangled sentiment and stance.<n>Political stance detection in low-resource and culturally complex settings poses a critical challenge for large language models.<n>We present ThaiFACTUAL, a lightweight, model-agnostic calibration framework that mitigates political bias without requiring fine-tuning.
arXiv Detail & Related papers (2025-09-26T06:26:21Z)
ANUBHUTI: A Comprehensive Corpus For Sentiment Analysis In Bangla Regional Languages [0.5062312533373298]
ANUBHUTI fills a critical gap in resources for sentiment analysis in low resource Bangla dialects.<n>The dataset features political and religious content, reflecting the contemporary socio political landscape of Bangladesh.<n>The dataset was further refined through systematic checks for missing data, anomalies, and inconsistencies.
arXiv Detail & Related papers (2025-06-26T18:13:54Z)
Profiling News Media for Factuality and Bias Using LLMs and the Fact-Checking Methodology of Human Experts [29.95198868148809]
We propose a novel methodology that emulates the criteria that professional fact-checkers use to assess the factuality and political bias of an entire outlet.<n>We provide an in-depth error analysis of the effect of media popularity and region on model performance.
arXiv Detail & Related papers (2025-06-14T15:49:20Z)
Geopolitical biases in LLMs: what are the "good" and the "bad" countries according to contemporary language models [52.00270888041742]
We introduce a novel dataset with neutral event descriptions and contrasting viewpoints from different countries.<n>Our findings show significant geopolitical biases, with models favoring specific national narratives.<n>Simple debiasing prompts had a limited effect on reducing these biases.
arXiv Detail & Related papers (2025-06-07T10:45:17Z)
Locating Information Gaps and Narrative Inconsistencies Across Languages: A Case Study of LGBT People Portrayals on Wikipedia [49.80565462746646]
We introduce the InfoGap method -- an efficient and reliable approach to locating information gaps and inconsistencies in articles at the fact level. We evaluate InfoGap by analyzing LGBT people's portrayals, across 2.7K biography pages on English, Russian, and French Wikipedias.
arXiv Detail & Related papers (2024-10-05T20:40:49Z)
Social Bias in Large Language Models For Bangla: An Empirical Study on Gender and Religious Bias [2.98683507969764]
It is important to assess the influence of different types of biases embedded in Large Language Models to ensure fair use in sensitive fields.<n>Although there have been extensive works on bias assessment in English, such efforts are rare and scarce for a major language like Bangla.<n>This is the first work of such kind involving bias assessment of LLMs for Bangla to the best of our knowledge.
arXiv Detail & Related papers (2024-07-03T22:45:36Z)
Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspective [66.34066553400108]
We conduct a rigorous evaluation of large language models' implicit bias towards certain demographics.<n>Inspired by psychometric principles, we propose three attack approaches, i.e., Disguise, Deception, and Teaching.<n>Our methods can elicit LLMs' inner bias more effectively than competitive baselines.
arXiv Detail & Related papers (2024-06-20T06:42:08Z)
ThatiAR: Subjectivity Detection in Arabic News Sentences [10.334164786614696]
This study presents the first large dataset for subjectivity detection in Arabic. It consists of 3.6K manually annotated sentences, and GPT-4o based explanation. We provide an in-depth analysis of the dataset, annotation process, and extensive benchmark results.
arXiv Detail & Related papers (2024-06-08T19:24:17Z)
Whose Side Are You On? Investigating the Political Stance of Large Language Models [56.883423489203786]
We investigate the political orientation of Large Language Models (LLMs) across a spectrum of eight polarizing topics. Our investigation delves into the political alignment of LLMs across a spectrum of eight polarizing topics, spanning from abortion to LGBTQ issues. The findings suggest that users should be mindful when crafting queries, and exercise caution in selecting neutral prompt language.
arXiv Detail & Related papers (2024-03-15T04:02:24Z)
Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis [86.49858739347412]
Large Language Models (LLMs) have sparked intense debate regarding the prevalence of bias in these models and its mitigation. We propose a prompt-based method for the extraction of confounding and mediating attributes which contribute to the decision process. We find that the observed disparate treatment can at least in part be attributed to confounding and mitigating attributes and model misalignment.
arXiv Detail & Related papers (2023-11-15T00:02:25Z)
Bias or Diversity? Unraveling Fine-Grained Thematic Discrepancy in U.S. News Headlines [63.52264764099532]
We use a large dataset of 1.8 million news headlines from major U.S. media outlets spanning from 2014 to 2022. We quantify the fine-grained thematic discrepancy related to four prominent topics - domestic politics, economic issues, social issues, and foreign affairs. Our findings indicate that on domestic politics and social issues, the discrepancy can be attributed to a certain degree of media bias.
arXiv Detail & Related papers (2023-03-28T03:31:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.