How Persuasive Could LLMs Be? A First Study Combining Linguistic-Rhetorical Analysis and User Experiments

• URL: http://arxiv.org/abs/2508.09614v1
• Date: Wed, 13 Aug 2025 08:45:04 GMT
• Title: How Persuasive Could LLMs Be? A First Study Combining Linguistic-Rhetorical Analysis and User Experiments
• Authors: Daniel Raffini, Agnese Macori, Lorenzo Porcaro, Tiziana Catarci, Marco Angelini,
• Abstract summary: The study finds that while participants often acknowledge the benefits highlighted by ChatGPT, ethical concerns tend to persist or even intensify post-interaction.<n>These findings highlight new insights on AI-generated persuasion in ethically sensitive domains and are a basis for future research.
• Score: 1.1277995582894218
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This study examines the rhetorical and linguistic features of argumentative texts generated by ChatGPT on ethically nuanced topics and investigates their persuasive impact on human readers.Through a user study involving 62 participants and pre-post interaction surveys, the paper analyzes how exposure to AI-generated arguments affects opinion change and user perception. A linguistic and rhetorical analysis of the generated texts reveals a consistent argumentative macrostructure, reliance on formulaic expressions, and limited stylistic richness. While ChatGPT demonstrates proficiency in constructing coherent argumentative texts, its persuasive efficacy appears constrained, particularly on topics involving ethical issues.The study finds that while participants often acknowledge the benefits highlighted by ChatGPT, ethical concerns tend to persist or even intensify post-interaction. The results also demonstrate a variation depending on the topic. These findings highlight new insights on AI-generated persuasion in ethically sensitive domains and are a basis for future research.

Related papers

• Detecting Winning Arguments with Large Language Models and Persuasion Strategies [7.089321248525487]
  This work investigates the role of persuasion strategies in determining the persuasiveness of a text.<n>We conduct experiments on three annotated argument datasets: Winning Arguments (built from the Change My View subreddit), Anthropic/Persuasion, and Persuasion for Good.<n>Results show that strategy-guided reasoning improves the prediction of persuasiveness.
  arXiv  Detail & Related papers  (2026-01-15T18:30:15Z)
• Persona Knowledge-Aligned Prompt Tuning Method for Online Debate [42.28019112668135]
  We propose a persona knowledge-aligned framework for argument quality assessment tasks from the audience side. This is the first work that leverages the emergence of ChatGPT and injects audience personae knowledge into smaller language models via prompt tuning.
  arXiv  Detail & Related papers  (2024-10-05T17:33:11Z)
• A Linguistic Comparison between Human and ChatGPT-Generated Conversations [9.022590646680095]
  The research employs Linguistic Inquiry and Word Count analysis, comparing ChatGPT-generated conversations with human conversations. Results show greater variability and authenticity in human dialogues, but ChatGPT excels in categories such as social processes, analytical style, cognition, attentional focus, and positive emotional tone.
  arXiv  Detail & Related papers  (2024-01-29T21:43:27Z)
• Natural Language Decompositions of Implicit Content Enable Better Text Representations [52.992875653864076]
  We introduce a method for the analysis of text that takes implicitly communicated content explicitly into account.<n>We use a large language model to produce sets of propositions that are inferentially related to the text that has been observed.<n>Our results suggest that modeling the meanings behind observed language, rather than the literal text alone, is a valuable direction for NLP.
  arXiv  Detail & Related papers  (2023-05-23T23:45:20Z)
• Uncovering the Potential of ChatGPT for Discourse Analysis in Dialogue: An Empirical Study [51.079100495163736]
  This paper systematically inspects ChatGPT's performance in two discourse analysis tasks: topic segmentation and discourse parsing. ChatGPT demonstrates proficiency in identifying topic structures in general-domain conversations yet struggles considerably in specific-domain conversations. Our deeper investigation indicates that ChatGPT can give more reasonable topic structures than human annotations but only linearly parses the hierarchical rhetorical structures.
  arXiv  Detail & Related papers  (2023-05-15T07:14:41Z)
• Consistency Analysis of ChatGPT [65.268245109828]
  This paper investigates the trustworthiness of ChatGPT and GPT-4 regarding logically consistent behaviour. Our findings suggest that while both models appear to show an enhanced language understanding and reasoning ability, they still frequently fall short of generating logically consistent predictions.
  arXiv  Detail & Related papers  (2023-03-11T01:19:01Z)
• Persua: A Visual Interactive System to Enhance the Persuasiveness of Arguments in Online Discussion [52.49981085431061]
  Enhancing people's ability to write persuasive arguments could contribute to the effectiveness and civility in online communication. We derived four design goals for a tool that helps users improve the persuasiveness of arguments in online discussions. Persua is an interactive visual system that provides example-based guidance on persuasive strategies to enhance the persuasiveness of arguments.
  arXiv  Detail & Related papers  (2022-04-16T08:07:53Z)
• Exploring Discourse Structures for Argument Impact Classification [48.909640432326654]
  This paper empirically shows that the discourse relations between two arguments along the context path are essential factors for identifying the persuasive power of an argument. We propose DisCOC to inject and fuse the sentence-level structural information with contextualized features derived from large-scale language models.
  arXiv  Detail & Related papers  (2021-06-02T06:49:19Z)
• The Role of Pragmatic and Discourse Context in Determining Argument Impact [39.70446357000737]
  This paper presents a new dataset to initiate the study of this aspect of argumentation. It consists of a diverse collection of arguments covering 741 controversial topics and comprising over 47,000 claims. We propose predictive models that incorporate the pragmatic and discourse context of argumentative claims and show that they outperform models that rely on claim-specific linguistic features for predicting the perceived impact of individual claims within a particular line of argument.
  arXiv  Detail & Related papers  (2020-04-06T23:00:37Z)
• What Changed Your Mind: The Roles of Dynamic Topics and Discourse in Argumentation Process [78.4766663287415]
  This paper presents a study that automatically analyzes the key factors in argument persuasiveness. We propose a novel neural model that is able to track the changes of latent topics and discourse in argumentative conversations.
  arXiv  Detail & Related papers  (2020-02-10T04:27:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.