The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents

• URL: http://arxiv.org/abs/2601.12164v1
• Date: Sat, 17 Jan 2026 21:00:36 GMT
• Title: The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents
• Abstract summary: This study presents an experimental comparison of large language models (LLMs) with prompts in Russian and Ukrainian.<n>Despite identical source material and parallel query structures, the resulting analyses varied substantially in rhetorical positioning, ideological orientation, and interpretive conclusions.<n>The Russian-language output echoed narratives common in Russian state discourse, characterizing civil society actors as illegitimate elites undermining democratic mandates.<n>The Ukrainian-language output adopted vocabulary characteristic of Western liberal-democratic political science, treating the same actors as legitimate stakeholders within democratic contestation.
• Score: 0.8528384027684192
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Large language models (LLMs) are increasingly deployed as analytical tools across multilingual contexts, yet their outputs may carry systematic biases conditioned by the language of the prompt. This study presents an experimental comparison of LLM-generated political analyses of a Ukrainian civil society document, using semantically equivalent prompts in Russian and Ukrainian. Despite identical source material and parallel query structures, the resulting analyses varied substantially in rhetorical positioning, ideological orientation, and interpretive conclusions. The Russian-language output echoed narratives common in Russian state discourse, characterizing civil society actors as illegitimate elites undermining democratic mandates. The Ukrainian-language output adopted vocabulary characteristic of Western liberal-democratic political science, treating the same actors as legitimate stakeholders within democratic contestation. These findings demonstrate that prompt language alone can produce systematically different ideological orientations from identical models analyzing identical content, with significant implications for AI deployment in polarized information environments, cross-lingual research applications, and the governance of AI systems in multilingual societies.

Related papers

• Language Ideologies in a Multilingual Society: An LLM-based Analysis of Luxembourgish News Comments [5.187863439550787]
  This paper explores the potential of large language models to assist in the detection of language ideologies.<n>We manually annotate a corpus of user comments in Luxembourgish with predefined ideological categories.<n>We also evaluate the performance of large language models under varying prompt conditions to assess their ability to replicate these human annotations.
  arXiv  Detail & Related papers  (2026-04-30T09:55:36Z)
• A Systematic Analysis of Biases in Large Language Models [40.23320093091831]
  Large language models (LLMs) have rapidly become indispensable tools for acquiring information and supporting human decision-making.<n>This study examines their underlying biases and inclinations across the dimensions of politics, ideology, alliance, language, and gender.<n>Results indicate that while the LLMs are aligned to be neutral and impartial, they still show biases and affinities of different types.
  arXiv  Detail & Related papers  (2025-12-16T03:38:08Z)
• Probing Social Identity Bias in Chinese LLMs with Gendered Pronouns and Social Groups [6.097454027594837]
  Large language models (LLMs) are increasingly deployed in user-facing applications, raising concerns about their potential to reflect and amplify social biases.<n>We investigate social identity framing in Chinese LLMs using Mandarin-specific prompts, evaluating responses to ingroup ("We") and outgroup ("They") framings, and extending the setting to 240 social groups salient in the Chinese context.<n>Across models, we observe systematic ingroup-positive and outgroup-negative tendencies, which are not confined to synthetic prompts but also appear in naturalistic dialogue.
  arXiv  Detail & Related papers  (2025-10-08T13:00:12Z)
• Artificial Authority: From Machine Minds to Political Alignments. An Experimental Analysis of Democratic and Autocratic Biases in Large-Language Models [0.11853986437641513]
  Political beliefs vary significantly across different countries, reflecting distinct historical, cultural, and institutional contexts.<n>The advent of generative artificial intelligence introduces new agents in the political space-agents trained on massive corpora.<n>This paper analyses whether Large Language Models (LLMs) display propensities consistent with democratic or autocratic world-views.
  arXiv  Detail & Related papers  (2025-09-29T11:26:58Z)
• Social Bias in Multilingual Language Models: A Survey [5.756606441319472]
  This systematic review analyzes emerging research that extends bias evaluation and mitigation approaches into multilingual and non-English contexts.<n>We examine these studies with respect to linguistic diversity, cultural awareness, and their choice of evaluation metrics and mitigation techniques.
  arXiv  Detail & Related papers  (2025-08-27T18:25:32Z)
• Framing Political Bias in Multilingual LLMs Across Pakistani Languages [6.5137518437747]
  We present a systematic evaluation of political bias in 13 state-of-the-art Large Language Models (LLMs) across five Pakistani languages.<n>Our framework integrates a culturally adapted Political Compass Test (PCT) with multi-level framing analysis.<n>Results show that while LLMs predominantly reflect liberal-left orientations consistent with Western training data, they exhibit more authoritarian framing in regional languages.
  arXiv  Detail & Related papers  (2025-05-29T15:15:42Z)
• Disparities in LLM Reasoning Accuracy and Explanations: A Case Study on African American English [66.97110551643722]
  We investigate dialectal disparities in Large Language Models (LLMs) reasoning tasks.<n>We find that LLMs produce less accurate responses and simpler reasoning chains and explanations for AAE inputs.<n>These findings highlight systematic differences in how LLMs process and reason about different language varieties.
  arXiv  Detail & Related papers  (2025-03-06T05:15:34Z)
• Large Language Models Reflect the Ideology of their Creators [71.65505524599888]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language.<n>This paper shows that the ideological stance of an LLM appears to reflect the worldview of its creators.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• Representation Bias in Political Sample Simulations with Large Language Models [54.48283690603358]
  This study seeks to identify and quantify biases in simulating political samples with Large Language Models. Using the GPT-3.5-Turbo model, we leverage data from the American National Election Studies, German Longitudinal Election Study, Zuobiao dataset, and China Family Panel Studies.
  arXiv  Detail & Related papers  (2024-07-16T05:52:26Z)
• Language Model Alignment in Multilingual Trolley Problems [138.5684081822807]
  Building on the Moral Machine experiment, we develop a cross-lingual corpus of moral dilemma vignettes in over 100 languages called MultiTP.<n>Our analysis explores the alignment of 19 different LLMs with human judgments, capturing preferences across six moral dimensions.<n>We discover significant variance in alignment across languages, challenging the assumption of uniform moral reasoning in AI systems.
  arXiv  Detail & Related papers  (2024-07-02T14:02:53Z)
• Large Language Models Meet Text-Centric Multimodal Sentiment Analysis: A Survey [66.166184609616]
  ChatGPT has opened up immense potential for applying large language models (LLMs) to text-centric multimodal tasks. It is still unclear how existing LLMs can adapt better to text-centric multimodal sentiment analysis tasks.
  arXiv  Detail & Related papers  (2024-06-12T10:36:27Z)
• CIVICS: Building a Dataset for Examining Culturally-Informed Values in Large Language Models [59.22460740026037]
  "CIVICS: Culturally-Informed & Values-Inclusive Corpus for Societal impacts" dataset is designed to evaluate the social and cultural variation of Large Language Models (LLMs) We create a hand-crafted, multilingual dataset of value-laden prompts which address specific socially sensitive topics, including LGBTQI rights, social welfare, immigration, disability rights, and surrogacy.
  arXiv  Detail & Related papers  (2024-05-22T20:19:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.