Large Language Models Are More Persuasive Than Incentivized Human Persuaders

• URL: http://arxiv.org/abs/2505.09662v2
• Date: Wed, 21 May 2025 13:29:57 GMT
• Title: Large Language Models Are More Persuasive Than Incentivized Human Persuaders
• Authors: Philipp Schoenegger, Francesco Salvi, Jiacheng Liu, Xiaoli Nan, Ramit Debnath, Barbara Fasolo, Evelina Leivada, Gabriel Recchia, Fritz Günther, Ali Zarifhonarvar, Joe Kwon, Zahoor Ul Islam, Marco Dehnert, Daryl Y. H. Lee, Madeline G. Reinecke, David G. Kamper, Mert Kobaş, Adam Sandford, Jonas Kgomo, Luke Hewitt, Shreya Kapoor, Kerem Oktar, Eyup Engin Kucuk, Bo Feng, Cameron R. Jones, Izzy Gainsburg, Sebastian Olschewski, Nora Heinzelmann, Francisco Cruz, Ben M. Tappin, Tao Ma, Peter S. Park, Rayan Onyonka, Arthur Hjorth, Peter Slattery, Qingcheng Zeng, Lennart Finke, Igor Grossmann, Alessandro Salatiello, Ezra Karger,
• Abstract summary: We compare the persuasion capabilities of a frontier large language model (LLM) against incentivized human persuaders in an online quiz setting.<n>LLM persuaders achieved significantly higher compliance with their directional persuasion attempts than incentivized human persuaders.<n>Our findings suggest that AI's persuasion capabilities already exceed those of humans that have real-money bonuses tied to performance.
• Score: 32.72764895955829
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We directly compare the persuasion capabilities of a frontier large language model (LLM; Claude Sonnet 3.5) against incentivized human persuaders in an interactive, real-time conversational quiz setting. In this preregistered, large-scale incentivized experiment, participants (quiz takers) completed an online quiz where persuaders (either humans or LLMs) attempted to persuade quiz takers toward correct or incorrect answers. We find that LLM persuaders achieved significantly higher compliance with their directional persuasion attempts than incentivized human persuaders, demonstrating superior persuasive capabilities in both truthful (toward correct answers) and deceptive (toward incorrect answers) contexts. We also find that LLM persuaders significantly increased quiz takers' accuracy, leading to higher earnings, when steering quiz takers toward correct answers, and significantly decreased their accuracy, leading to lower earnings, when steering them toward incorrect answers. Overall, our findings suggest that AI's persuasion capabilities already exceed those of humans that have real-money bonuses tied to performance. Our findings of increasingly capable AI persuaders thus underscore the urgency of emerging alignment and governance frameworks.

Related papers

• MMPersuade: A Dataset and Evaluation Framework for Multimodal Persuasion [73.99171322670772]
  Large Vision-Language Models (LVLMs) are increasingly deployed in domains such as shopping, health, and news.<n> MMPersuade provides a unified framework for systematically studying multimodal persuasion dynamics in LVLMs.
  arXiv  Detail & Related papers  (2025-10-26T17:39:21Z)
• When AI Gets Persuaded, Humans Follow: Inducing the Conformity Effect in Persuasive Dialogue [0.8594140167290097]
  This study introduces a "Persuadee Agent" that is persuaded alongside a human participant.<n>We conducted a text-based dialogue experiment with human participants.<n>When the Persuadee Agent accepted persuasion, both perceived persuasiveness and actual attitude change significantly improved.
  arXiv  Detail & Related papers  (2025-10-05T14:37:46Z)
• Can You Trick the Grader? Adversarial Persuasion of LLM Judges [15.386741140145205]
  This study is the first to reveal that strategically embedded persuasive language can bias LLM judges when scoring mathematical reasoning tasks.<n>We formalize seven persuasion techniques (Majority, Consistency, Flattery, Reciprocity, Pity, Authority, Identity) and embed them into otherwise identical responses.<n>We find that persuasive language leads LLM judges to assign inflated scores to incorrect solutions, by up to 8% on average, with Consistency causing the most severe distortion.
  arXiv  Detail & Related papers  (2025-08-11T09:45:02Z)
• The Levers of Political Persuasion with Conversational AI [4.6244198651412045]
  There are widespread fears that conversational AI could soon exert unprecedented influence over human beliefs.<n>We show that the persuasive power of current and near-future AI is likely to stem more from post-training and prompting methods.
  arXiv  Detail & Related papers  (2025-07-18T13:50:09Z)
• AI Debate Aids Assessment of Controversial Claims [86.47978525513236]
  We study whether AI debate can guide biased judges toward the truth by having two AI systems debate opposing sides of controversial COVID-19 factuality claims.<n>In our human study, we find that debate-where two AI advisor systems present opposing evidence-based arguments-consistently improves judgment accuracy and confidence calibration.<n>In our AI judge study, we find that AI judges with human-like personas achieve even higher accuracy (78.5%) than human judges (70.1%) and default AI judges without personas (69.8%)
  arXiv  Detail & Related papers  (2025-06-02T19:01:53Z)
• Must Read: A Systematic Survey of Computational Persuasion [60.83151988635103]
  AI-driven persuasion can be leveraged for beneficial applications, but also poses threats through manipulation and unethical influence.<n>Our survey outlines future research directions to enhance the safety, fairness, and effectiveness of AI-powered persuasion.
  arXiv  Detail & Related papers  (2025-05-12T17:26:31Z)
• Persuade Me if You Can: A Framework for Evaluating Persuasion Effectiveness and Susceptibility Among Large Language Models [9.402740034754455]
  Large Language Models (LLMs) demonstrate persuasive capabilities that rival human-level persuasion.<n>LLMs' susceptibility to persuasion raises concerns about alignment with ethical principles.<n>We introduce Persuade Me If You Can (PMIYC), an automated framework for evaluating persuasion through multi-agent interactions.
  arXiv  Detail & Related papers  (2025-03-03T18:53:21Z)
• Persuasion Should be Double-Blind: A Multi-Domain Dialogue Dataset With Faithfulness Based on Causal Theory of Mind [21.022976907694265]
  Recent persuasive dialogue datasets often fail to align with real-world interpersonal interactions.<n>We introduce ToMMA, a novel multi-agent framework for dialogue generation guided by causal Theory of Mind.<n>We present CToMPersu, a multi-domain, multi-turn persuasive dialogue dataset.
  arXiv  Detail & Related papers  (2025-02-28T18:28:16Z)
• Mind What You Ask For: Emotional and Rational Faces of Persuasion by Large Language Models [0.0]
  Large language models (LLMs) are increasingly effective at persuading us that their answers are valuable.<n>This study examines what are the psycholinguistic features of the responses used by twelve different language models.<n>We ask whether and how we can mitigate the risks of LLM-driven mass misinformation.
  arXiv  Detail & Related papers  (2025-02-13T15:15:53Z)
• Interactive Dialogue Agents via Reinforcement Learning on Hindsight Regenerations [58.65755268815283]
  Many real dialogues are interactive, meaning an agent's utterances will influence their conversational partner, elicit information, or change their opinion. We use this fact to rewrite and augment existing suboptimal data, and train via offline reinforcement learning (RL) an agent that outperforms both prompting and learning from unaltered human demonstrations. Our results in a user study with real humans show that our approach greatly outperforms existing state-of-the-art dialogue agents.
  arXiv  Detail & Related papers  (2024-11-07T21:37:51Z)
• Zero-shot Persuasive Chatbots with LLM-Generated Strategies and Information Retrieval [7.925754291635035]
  Persuasion plays a pivotal role in a wide range of applications from health intervention to the promotion of social good. Persuasive chatbots employed responsibly for social good can be an enabler of positive individual and social change. We propose PersuaBot, a zero-shot chatbots based on Large Language Models (LLMs) that is factual and more persuasive by leveraging many more nuanced strategies. Our experiments on simulated and human conversations show that our zero-shot approach is more persuasive than prior work, while achieving factual accuracy surpassing state-of-the-art knowledge-oriented chatbots.
  arXiv  Detail & Related papers  (2024-07-04T02:28:21Z)
• "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust [51.542856739181474]
  We show how different natural language expressions of uncertainty impact participants' reliance, trust, and overall task performance. We find that first-person expressions decrease participants' confidence in the system and tendency to agree with the system's answers, while increasing participants' accuracy. Our findings suggest that using natural language expressions of uncertainty may be an effective approach for reducing overreliance on LLMs, but that the precise language used matters.
  arXiv  Detail & Related papers  (2024-05-01T16:43:55Z)
• What Changed Your Mind: The Roles of Dynamic Topics and Discourse in Argumentation Process [78.4766663287415]
  This paper presents a study that automatically analyzes the key factors in argument persuasiveness. We propose a novel neural model that is able to track the changes of latent topics and discourse in argumentative conversations.
  arXiv  Detail & Related papers  (2020-02-10T04:27:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.