Political Bias Audits of LLMs Capture Sycophancy to the Inferred Auditor

• URL: http://arxiv.org/abs/2604.27633v1
• Date: Thu, 30 Apr 2026 09:23:00 GMT
• Title: Political Bias Audits of LLMs Capture Sycophancy to the Inferred Auditor
• Abstract summary: Large language models (LLMs) are commonly evaluated for political bias based on their responses to fixed questionnaires.<n>We show that standard political-bias audits partly capture sycophantic accommodation to the inferred auditor.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are commonly evaluated for political bias based on their responses to fixed questionnaires, which typically place frontier models on the political left. A parallel literature shows that LLMs are sycophantic: they adapt their answers to the views, identities, and expectations of the user. We show that these findings are linked: standard political-bias audits partly capture sycophantic accommodation to the inferred auditor. We employ a factorial experiment across three major audit instruments--the Political Compass Test, the Pew Political Typology, and 1,540 partisan-benchmarked Pew American Trends Panel items--administered to six frontier LLMs while varying only the asker's stated identity (N = 30,990 responses). At baseline, all six models lean left. When the asker identifies as a conservative Republican, responses shift sharply: the share of items closer to Democrats falls by 28-62 percentage points, and all six models move right of center. A mirror-image progressive-Democrat cue produces little change; rightward accommodation is 8.0$\times$ larger than leftward. When asked who the default asker is, models identify an auditor, researcher, or academic; when asked what answer that asker expects, they select the Democrat-coded option 75% of the time, nearly the rate under an explicit progressive cue. These patterns are inconsistent with a purely fixed model ideology and indicate that single-prompt audits capture an interaction between model and inferred interlocutor. Political bias in LLMs is therefore not a fixed point on an ideological scale but a response profile that must be mapped across realistic interlocutors.

Related papers

• Auditing Alignment Controllability in LLMs via Political Axes [0.0]
  Political audits usually reduce each to one point on a political compass.<n>What counts is how far, and in which directions, a model's answers can be steered.<n>We run a dispersion-first stress test of prompt-based controllability across 12 ideological personas.
  arXiv  Detail & Related papers  (2026-07-26T07:38:11Z)
• The Invisible Coalition Partner: How LLMs Vote When Democracy Gets Concrete [0.0]
  Abstract questionnaires do not predict concrete behavior.<n>For some models, the language of a political question changes the answer more than the political content does.<n>Two models exhibit systematic change-aversion rather than political bias.
  arXiv  Detail & Related papers  (2026-05-03T08:48:33Z)
• PoliticsBench: Benchmarking Political Values in Large Language Models with Multi-Turn Roleplay [0.5729426778193398]
  This study investigates political bias in eight prominent Large Language Models (LLMs) using PoliticsBench.<n>We test whether commercially developed LLMs display a systematic left-leaning bias that becomes more pronounced in later stages of roleplay.
  arXiv  Detail & Related papers  (2026-03-25T01:54:56Z)
• Generalization to Political Beliefs from Fine-Tuning on Sports Team Preferences [0.0]
  We present results in which an LLM fine-tuned to prefer either coastal sports teams or Southern sports teams adopt political beliefs that diverge significantly from those of the base model.<n>We find that their responses are usually similar to each other, without a clear-cut liberal or conservative bias.
  arXiv  Detail & Related papers  (2026-01-07T20:18:19Z)
• PolBiX: Detecting LLMs' Political Bias in Fact-Checking through X-phemisms [1.0439136407307046]
  We investigate political bias through exchanging words with euphemisms or dysphemisms in German claims.<n>We find that, more than political leaning, the presence of judgmental words significantly influences truthfulness assessment.
  arXiv  Detail & Related papers  (2025-09-18T18:26:53Z)
• "Amazing, They All Lean Left" -- Analyzing the Political Temperaments of Current LLMs [5.754220850145368]
  We find strong and consistent prioritization of liberal-leaning values, particularly care and fairness, across most models.<n>We argue that this "liberal tilt" is not a programming error but an emergent property of training on democratic rights-focused discourse.<n>Rather than undermining democratic discourse, this pattern may offer a new lens through which to examine collective reasoning.
  arXiv  Detail & Related papers  (2025-07-08T21:19:25Z)
• Democratic or Authoritarian? Probing a New Dimension of Political Biases in Large Language Models [72.89977583150748]
  We propose a novel methodology to assess how Large Language Models align with broader geopolitical value systems.<n>We find that LLMs generally favor democratic values and leaders, but exhibit increases favorability toward authoritarian figures when prompted in Mandarin.
  arXiv  Detail & Related papers  (2025-06-15T07:52:07Z)
• Surface Fairness, Deep Bias: A Comparative Study of Bias in Language Models [45.41676783204022]
  We investigate various proxy measures of bias in large language models (LLMs)<n>We find that evaluating models with pre-prompted personae on a multi-subject benchmark (MMLU) leads to negligible and mostly random differences in scores.<n>With the recent trend for LLM assistant memory and personalization, these problems open up from a different angle.
  arXiv  Detail & Related papers  (2025-06-12T08:47:40Z)
• Large Language Models Reflect the Ideology of their Creators [71.65505524599888]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language.<n>This paper shows that the ideological stance of an LLM appears to reflect the worldview of its creators.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• Whose Side Are You On? Investigating the Political Stance of Large Language Models [56.883423489203786]
  We investigate the political orientation of Large Language Models (LLMs) across a spectrum of eight polarizing topics. Our investigation delves into the political alignment of LLMs across a spectrum of eight polarizing topics, spanning from abortion to LGBTQ issues. The findings suggest that users should be mindful when crafting queries, and exercise caution in selecting neutral prompt language.
  arXiv  Detail & Related papers  (2024-03-15T04:02:24Z)
• Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models [61.45529177682614]
  We challenge the prevailing constrained evaluation paradigm for values and opinions in large language models. We show that models give substantively different answers when not forced. We distill these findings into recommendations and open challenges in evaluating values and opinions in LLMs.
  arXiv  Detail & Related papers  (2024-02-26T18:00:49Z)
• What Do Llamas Really Think? Revealing Preference Biases in Language Model Representations [62.91799637259657]
  Do large language models (LLMs) exhibit sociodemographic biases, even when they decline to respond? We study this research question by probing contextualized embeddings and exploring whether this bias is encoded in its latent representations. We propose a logistic Bradley-Terry probe which predicts word pair preferences of LLMs from the words' hidden vectors.
  arXiv  Detail & Related papers  (2023-11-30T18:53:13Z)
• Whose Opinions Do Language Models Reflect? [88.35520051971538]
  We investigate the opinions reflected by language models (LMs) by leveraging high-quality public opinion polls and their associated human responses. We find substantial misalignment between the views reflected by current LMs and those of US demographic groups. Our analysis confirms prior observations about the left-leaning tendencies of some human feedback-tuned LMs.
  arXiv  Detail & Related papers  (2023-03-30T17:17:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.