Benchmarking Political Persuasion Risks Across Frontier Large Language Models

• URL: http://arxiv.org/abs/2603.09884v1
• Date: Tue, 10 Mar 2026 16:42:05 GMT
• Title: Benchmarking Political Persuasion Risks Across Frontier Large Language Models
• Abstract summary: Prior research has claimed that Large Language Models (LLMs) are not more persuasive than standard political campaign practices.<n>We evaluate seven state-of-the-art LLMs developed by Anthropic, OpenAI, Google, and xAI.<n>Specifically, Claude models exhibit the highest persuasiveness, while Grok exhibits the lowest.
• Score: 1.8137773774884758
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Concerns persist regarding the capacity of Large Language Models (LLMs) to sway political views. Although prior research has claimed that LLMs are not more persuasive than standard political campaign practices, the recent rise of frontier models warrants further study. In two survey experiments (N=19,145) across bipartisan issues and stances, we evaluate seven state-of-the-art LLMs developed by Anthropic, OpenAI, Google, and xAI. We find that LLMs outperform standard campaign advertisements, with heterogeneity in performance across models. Specifically, Claude models exhibit the highest persuasiveness, while Grok exhibits the lowest. The results are robust across issues and stances. Moreover, in contrast to the findings in Hackenburg et al. (2025b) and Lin et al. (2025) that information-based prompts boost persuasiveness, we find that the effectiveness of information-based prompts is model-dependent: they increase the persuasiveness of Claude and Grok while substantially reducing that of GPT. We introduce a data-driven and strategy-agnostic LLM-assisted conversation analysis approach to identify and assess underlying persuasive strategies. Our work benchmarks the persuasive risks of frontier models and provides a framework for cross-model comparative risk assessment.

Related papers

• Reasoning Boosts Opinion Alignment in LLMs [30.074886272342777]
  We study whether reasoning can improve opinion alignment.<n>Motivated by the recent advancement in mathematical reasoning enabled by reinforcement learning (RL), we train models to produce profile-consistent answers.<n>We evaluate our approach on three datasets covering U.S., European, and Swiss politics.
  arXiv  Detail & Related papers  (2026-03-01T18:19:44Z)
• Identifying Fine-grained Forms of Populism in Political Discourse: A Case Study on Donald Trump's Presidential Campaigns [17.242754326635833]
  This paper examines whether Large Language Models can identify and classify fine-grained forms of populism.<n>We evaluate a range of pre-trained (large) language models, both open-weight and proprietary, across multiple prompting paradigms.<n>We find that a fine-tuned RoBERTa classifier vastly outperforms all new-era instruction-tuned LLMs, unless fine-tuned.
  arXiv  Detail & Related papers  (2025-07-25T14:18:54Z)
• Beyond Reactive Safety: Risk-Aware LLM Alignment via Long-Horizon Simulation [69.63626052852153]
  We propose a proof-of-concept framework that projects how model-generated advice could propagate through societal systems.<n>We also introduce a dataset of 100 indirect harm scenarios, testing models' ability to foresee adverse, non-obvious outcomes from seemingly harmless user prompts.
  arXiv  Detail & Related papers  (2025-06-26T02:28:58Z)
• A Framework to Assess the Persuasion Risks Large Language Model Chatbots Pose to Democratic Societies [1.1819975950139372]
  Large Language Models (LLMs) pose to democratic societies through their persuasive capabilities.<n>We conduct two survey experiments and a real-world simulation exercise to determine whether it is more cost effective to persuade a large number of voters using LLMs.<n>We estimate that LLM-based persuasion costs between $48-$74 per persuaded voter compared to $100 for traditional campaign methods.
  arXiv  Detail & Related papers  (2025-04-29T16:02:51Z)
• Fact-checking with Generative AI: A Systematic Cross-Topic Examination of LLMs Capacity to Detect Veracity of Political Information [0.0]
  The purpose of this study is to assess how large language models (LLMs) can be used for fact-checking.<n>We use AI auditing methodology that systematically evaluates performance of five LLMs.<n>The results indicate that models are better at identifying false statements, especially on sensitive topics.
  arXiv  Detail & Related papers  (2025-03-11T13:06:40Z)
• Among Them: A game-based framework for assessing persuasion capabilities of LLMs [0.8763629723457529]
  Large language models (LLMs) and autonomous AI agents have raised concerns about their potential for automated persuasion and social influence.<n>We present an Among Us-inspired game framework for assessing LLM deception skills in a controlled environment.
  arXiv  Detail & Related papers  (2025-02-27T12:26:21Z)
• LlaMADRS: Prompting Large Language Models for Interview-Based Depression Assessment [75.44934940580112]
  This study introduces LlaMADRS, a novel framework leveraging open-source Large Language Models (LLMs) to automate depression severity assessment.<n>We employ a zero-shot prompting strategy with carefully designed cues to guide the model in interpreting and scoring transcribed clinical interviews.<n>Our approach, tested on 236 real-world interviews, demonstrates strong correlations with clinician assessments.
  arXiv  Detail & Related papers  (2025-01-07T08:49:04Z)
• Persuasion with Large Language Models: a Survey [49.86930318312291]
  Large Language Models (LLMs) have created new disruptive possibilities for persuasive communication. In areas such as politics, marketing, public health, e-commerce, and charitable giving, such LLM Systems have already achieved human-level or even super-human persuasiveness. Our survey suggests that the current and future potential of LLM-based persuasion poses profound ethical and societal risks.
  arXiv  Detail & Related papers  (2024-11-11T10:05:52Z)
• Large Language Models Reflect the Ideology of their Creators [71.65505524599888]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language.<n>This paper shows that the ideological stance of an LLM appears to reflect the worldview of its creators.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspective [66.34066553400108]
  We conduct a rigorous evaluation of large language models' implicit bias towards certain demographics.<n>Inspired by psychometric principles, we propose three attack approaches, i.e., Disguise, Deception, and Teaching.<n>Our methods can elicit LLMs' inner bias more effectively than competitive baselines.
  arXiv  Detail & Related papers  (2024-06-20T06:42:08Z)
• Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis [86.49858739347412]
  Large Language Models (LLMs) have sparked intense debate regarding the prevalence of bias in these models and its mitigation. We propose a prompt-based method for the extraction of confounding and mediating attributes which contribute to the decision process. We find that the observed disparate treatment can at least in part be attributed to confounding and mitigating attributes and model misalignment.
  arXiv  Detail & Related papers  (2023-11-15T00:02:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.