Evaluating the Capabilities of LLMs for Persuasive Dialogue

• URL: http://arxiv.org/abs/2608.29738v1
• Date: Sun, 30 Aug 2026 12:03:59 GMT
• Title: Evaluating the Capabilities of LLMs for Persuasive Dialogue
• Abstract summary: Large language models (LLMs) can generate apparently highly persuasive text, but does sounding persuasive mean arguing well?<n>We introduce textscPersuasio, a multi-agent dialogue platform grounded in a formal argumentation-based theory of persuasion dialogues.<n>Using this system, we generated 192 debates on a UK political topic between humans and LLMs, and evaluated 22 interlocutors through both automated adjudication and 9,702 crowdsourced pairwise judgements.
• Score: 3.3571381688392488
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) can generate apparently highly persuasive text, but does sounding persuasive mean arguing well? We introduce \textsc{Persuasio}, a multi-agent dialogue platform grounded in a formal argumentation-based theory of persuasion dialogues that adjudicates logical winners during free-text debates. Using this system, we generated 192 debates on a UK political topic between humans and LLMs, and evaluated 22 interlocutors through both automated adjudication and 9,702 crowdsourced pairwise judgements across 1{,}386 annotation instances. We observed a consistent decoupling between subjective and formal persuasiveness: LLMs dominated the subjective ranking yet performed substantially worse under argumentation-theoretic adjudication, where humans remained competitive. Multi-agent and retrieval-augmented variants further widened this divergence. These findings reveal a systematic gap between rhetorical fluency and formal argumentative strength in LLM-based persuasive dialogues.

Related papers

• SAD: A Large-Scale Strategic Argumentative Dialogue Dataset [60.33125467375306]
  In practice, argumentation is often realized as multi-turn dialogue.<n>We present the first large-scale textbfStrategic textbfArgumentative textbfDialogue dataset, consisting of 392,822 examples.
  arXiv  Detail & Related papers  (2026-01-12T11:11:37Z)
• MMPersuade: A Dataset and Evaluation Framework for Multimodal Persuasion [73.99171322670772]
  Large Vision-Language Models (LVLMs) are increasingly deployed in domains such as shopping, health, and news.<n> MMPersuade provides a unified framework for systematically studying multimodal persuasion dynamics in LVLMs.
  arXiv  Detail & Related papers  (2025-10-26T17:39:21Z)
• The Thin Line Between Comprehension and Persuasion in LLMs [0.0]
  Large language models (LLMs) are excellent at maintaining high-level, convincing dialogues.<n>We measure how this capability relates to their understanding of what is being talked about.<n>We find that LLMs are capable of maintaining coherent, persuasive debates, often swaying the beliefs of participants and audiences alike.
  arXiv  Detail & Related papers  (2025-07-02T17:46:56Z)
• DialogueReason: Rule-Based RL Sparks Dialogue Reasoning in LLMs [54.4857963044859]
  We propose DialogueReason, a reasoning paradigm that uncovers the lost roles in monologue-style reasoning models.<n>Our work consists of an analysis of monologue reasoning patterns and the development of a dialogue-based reasoning approach.
  arXiv  Detail & Related papers  (2025-05-11T16:39:58Z)
• Large Language Models Understanding: an Inherent Ambiguity Barrier [0.0]
  A lively ongoing debate is taking place, since the emergence of Large Language Models (LLMs) with regards to their capability to understand the world.<n>Arguments and counter-arguments have been proposed based upon thought experiments, anecdotal conversations between LLMs and humans, statistical linguistic analysis, philosophical considerations, and more.<n>In this brief paper we present a counter-argument based upon a thought experiment and semi-formal considerations leading to an inherent ambiguity barrier.
  arXiv  Detail & Related papers  (2025-05-01T16:55:44Z)
• Can LLMs Understand the Implication of Emphasized Sentences in Dialogue? [64.72966061510375]
  Emphasis is a crucial component in human communication, which indicates the speaker's intention and implication beyond pure text in dialogue. This paper introduces Emphasized-Talk, a benchmark with emphasis-annotated dialogue samples capturing the implications of emphasis. We evaluate various Large Language Models (LLMs), both open-source and commercial, to measure their performance in understanding emphasis.
  arXiv  Detail & Related papers  (2024-06-16T20:41:44Z)
• SocraSynth: Multi-LLM Reasoning with Conditional Statistics [2.5200794639628032]
  Large language models (LLMs) face criticisms for biases, hallucinations, and a lack of reasoning capability. This paper introduces Socra Synth, a multi-LLM agent reasoning platform developed to mitigate these issues.
  arXiv  Detail & Related papers  (2024-01-19T07:16:21Z)
• BotChat: Evaluating LLMs' Capabilities of Having Multi-Turn Dialogues [72.65163468440434]
  This report provides a preliminary evaluation of existing large language models for human-style multi-turn chatting. We prompt large language models (LLMs) to generate a full multi-turn dialogue based on the ChatSEED, utterance by utterance. We find GPT-4 can generate human-style multi-turn dialogues with impressive quality, significantly outperforms its counterparts.
  arXiv  Detail & Related papers  (2023-10-20T16:53:51Z)
• A Unifying Framework for Learning Argumentation Semantics [47.84663434179473]
  We present a novel framework, which uses an Inductive Logic Programming approach to learn the acceptability semantics for several abstract and structured argumentation frameworks in an interpretable way.<n>Our framework outperforms existing argumentation solvers, thus opening up new future research directions in the area of formal argumentation and human-machine dialogues.
  arXiv  Detail & Related papers  (2023-10-18T20:18:05Z)
• Opening up Minds with Argumentative Dialogues [14.903849413791443]
  argumentative dialogues aim to open up people's minds to help them become more understanding to views that are unfamiliar or in opposition to their own convictions. We present a dataset of 183 argumentative dialogues about 3 controversial topics: veganism, Brexit and COVID-19 vaccination. Open-mindedness is measured before and after engaging in the dialogue using a questionnaire from the psychology literature.
  arXiv  Detail & Related papers  (2023-01-16T12:47:16Z)
• Strategic Argumentation Dialogues for Persuasion: Framework and Experiments Based on Modelling the Beliefs and Concerns of the Persuadee [6.091096843566857]
  Two key dimensions for determining whether an argument is good in a particular dialogue are the degree to which the intended audience believes the argument and counterarguments, and the impact that the argument has on the concerns of the intended audience. We present a framework for modelling persuadees in terms of their beliefs and concerns, and for harnessing these models in optimizing the choice of move in persuasion dialogues.
  arXiv  Detail & Related papers  (2021-01-28T08:49:24Z)
• Rethinking Dialogue State Tracking with Reasoning [76.0991910623001]
  This paper proposes to track dialogue states gradually with reasoning over dialogue turns with the help of the back-end data. Empirical results demonstrate that our method significantly outperforms the state-of-the-art methods by 38.6% in terms of joint belief accuracy for MultiWOZ 2.1.
  arXiv  Detail & Related papers  (2020-05-27T02:05:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.