How Identity and Opinion Shape Political Sycophancy in LLMs

• URL: http://arxiv.org/abs/2608.29198v1
• Date: Sat, 29 Aug 2026 11:12:03 GMT
• Title: How Identity and Opinion Shape Political Sycophancy in LLMs
• Abstract summary: We introduce a framework that disentangles two distinct triggers of political sycophancy: opinion and identity.<n>Using 450 manually-checked political dilemmas as controlled probes, we evaluate 13 instruction-tuned Large Language Models.<n>Our results suggest that LLM political stance is interactively and steerably vulnerable rather than being a fixed trait.
• Score: 17.0131514909503
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As Large Language Models (LLMs) increasingly encourage users to disclose personal profiles for tailored assistance, measuring their political alignment becomes increasingly important. However, many existing benchmarks for assessing political behavior rely on closed-ended questions and do not fully capture how a model's stance may adapt to user-provided context during interaction. We introduce a framework that disentangles two distinct triggers of political sycophancy: opinion (aligning with explicit narratives) and identity (stereotyping based on demographic labels). Using 450 manually-checked political dilemmas as controlled probes, we evaluate 13 instruction-tuned LLMs. We uncover a dissociation: a model's susceptibility to explicit opinions does not necessarily predict its susceptibility to identity cues, and vice versa. When both signals are present, their effects are generally sub-additive rather than simply additive. Additionally, system-level personas primarily shift a model's baseline stance while having limited effect on the stance shift caused by user opinion or identity. Ultimately, our results suggest that LLM political stance is interactively and steerably vulnerable rather than being a fixed trait, highlighting how personalization may amplify identity- or opinion-conditioned shifts in the model's behaviors.

Related papers

• Personalization Increases Affective Alignment but Has Role-Dependent Effects on Epistemic Independence in LLMs [2.1198879079315573]
  Large Language Models (LLMs) are prone to sycophantic behavior, uncritically conforming to user beliefs.<n>We present a rigorous evaluation of personalization's impact on LLM sycophancy across nine frontier models and five benchmark datasets.
  arXiv  Detail & Related papers  (2026-02-03T19:04:03Z)
• Persona Prompting as a Lens on LLM Social Reasoning [5.001433675691563]
  For socially sensitive tasks like hate speech detection, the quality of explanations from Large Language Models (LLMs) is crucial.<n>While Persona prompting (PP) is increasingly used as a way to steer model towards user-specific generation, its effect on model rationales remains underexplored.
  arXiv  Detail & Related papers  (2026-01-28T16:41:17Z)
• Demographic Probing of Large Language Models Lacks Construct Validity [16.29607362682272]
  We study how large language models adapt their behavior to demographic attributes.<n>This approach typically uses a single demographic cue in isolation as a signal for group membership.<n>We find that cues intended to represent the same demographic group induce only partially overlapping changes in model behavior.
  arXiv  Detail & Related papers  (2026-01-26T13:41:35Z)
• Revisiting LLM Value Probing Strategies: Are They Robust and Expressive? [81.49470136653665]
  We evaluate the robustness and expressiveness of value representations across three widely used probing strategies.<n>We show that the demographic context has little effect on the free-text generation, and the models' values only weakly correlate with their preference for value-based actions.
  arXiv  Detail & Related papers  (2025-07-17T18:56:41Z)
• Persona-driven Simulation of Voting Behavior in the European Parliament with Large Language Models [3.217354895187878]
  We analyze whether zero-shot persona prompting with limited information can accurately predict individual voting decisions.<n>We find that we can simulate voting behavior of Members of the European Parliament reasonably well with a weighted F1 score of approximately 0.793.
  arXiv  Detail & Related papers  (2025-06-13T14:02:21Z)
• Mapping and Influencing the Political Ideology of Large Language Models using Synthetic Personas [5.237116285113809]
  We map the political distribution of persona-based prompted large language models using the Political Compass Test (PCT)<n>Our experiments reveal that synthetic personas predominantly cluster in the left-libertarian quadrant, with models demonstrating varying degrees of responsiveness when prompted with explicit ideological descriptors.<n>While all models demonstrate significant shifts towards right-authoritarian positions, they exhibit more limited shifts towards left-libertarian positions, suggesting an asymmetric response to ideological manipulation that may reflect inherent biases in model training.
  arXiv  Detail & Related papers  (2024-12-19T13:36:18Z)
• Stereotype or Personalization? User Identity Biases Chatbot Recommendations [65.10885980789293]
  We show that models generate racially stereotypical recommendations regardless of whether the user revealed their identity intentionally.<n>We argue that chatbots ought to transparently indicate when recommendations are influenced by a user's revealed identity characteristics, but observe that they currently fail to do so.
  arXiv  Detail & Related papers  (2024-10-08T01:51:55Z)
• Whose Side Are You On? Investigating the Political Stance of Large Language Models [56.883423489203786]
  We investigate the political orientation of Large Language Models (LLMs) across a spectrum of eight polarizing topics. Our investigation delves into the political alignment of LLMs across a spectrum of eight polarizing topics, spanning from abortion to LGBTQ issues. The findings suggest that users should be mindful when crafting queries, and exercise caution in selecting neutral prompt language.
  arXiv  Detail & Related papers  (2024-03-15T04:02:24Z)
• Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models [61.45529177682614]
  We challenge the prevailing constrained evaluation paradigm for values and opinions in large language models. We show that models give substantively different answers when not forced. We distill these findings into recommendations and open challenges in evaluating values and opinions in LLMs.
  arXiv  Detail & Related papers  (2024-02-26T18:00:49Z)
• On the steerability of large language models toward data-driven personas [98.9138902560793]
  Large language models (LLMs) are known to generate biased responses where the opinions of certain groups and populations are underrepresented. Here, we present a novel approach to achieve controllable generation of specific viewpoints using LLMs.
  arXiv  Detail & Related papers  (2023-11-08T19:01:13Z)
• Face Off: Polarized Public Opinions on Personal Face Mask Usage during the COVID-19 Pandemic [77.34726150561087]
  A series of policy shifts by various governmental bodies have been speculated to have contributed to the polarization of face masks. We propose a novel approach to accurately gauge public sentiment towards face masks in the United States during COVID-19. We find two key policy-shift events contributed to statistically significant changes in sentiment for both Republicans and Democrats.
  arXiv  Detail & Related papers  (2020-10-31T18:52:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.