Analysing Differences in Persuasive Language in LLM-Generated Text: Uncovering Stereotypical Gender Patterns

• URL: http://arxiv.org/abs/2601.05751v1
• Date: Fri, 09 Jan 2026 12:07:38 GMT
• Title: Analysing Differences in Persuasive Language in LLM-Generated Text: Uncovering Stereotypical Gender Patterns
• Authors: Amalie Brogaard Pauli, Maria Barrett, Max Müller-Eberstein, Isabelle Augenstein, Ira Assent,
• Abstract summary: We evaluate how persuasive language generation is affected by recipient gender, sender intent, or output language.<n>Our results reveal significant gender differences in the persuasive language generated across all models.
• Score: 35.516436178105
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are increasingly used for everyday communication tasks, including drafting interpersonal messages intended to influence and persuade. Prior work has shown that LLMs can successfully persuade humans and amplify persuasive language. It is therefore essential to understand how user instructions affect the generation of persuasive language, and to understand whether the generated persuasive language differs, for example, when targeting different groups. In this work, we propose a framework for evaluating how persuasive language generation is affected by recipient gender, sender intent, or output language. We evaluate 13 LLMs and 16 languages using pairwise prompt instructions. We evaluate model responses on 19 categories of persuasive language using an LLM-as-judge setup grounded in social psychology and communication science. Our results reveal significant gender differences in the persuasive language generated across all models. These patterns reflect biases consistent with gender-stereotypical linguistic tendencies documented in social psychology and sociolinguistics.

Related papers

• Can AI-Generated Persuasion Be Detected? Persuaficial Benchmark and AI vs. Human Linguistic Differences [5.782702968306668]
  Large Language Models (LLMs) can generate highly persuasive text, raising concerns about their misuse for propaganda, manipulation, and other harmful purposes.<n>We categorize controllable generation approaches for producing persuasive content with LLMs and introduce Persuaficial, a high-quality multilingual benchmark covering six languages.<n>We find that although overtly persuasive LLM-generated texts can be easier to detect than human-written ones, subtle LLM-generated persuasion consistently degrades automatic detection performance.
  arXiv  Detail & Related papers  (2026-01-08T13:22:25Z)
• EuroGEST: Investigating gender stereotypes in multilingual language models [58.871032460235575]
  We introduce EuroGEST, a dataset designed to measure gender-stereotypical reasoning in LLMs across English and 29 European languages.<n>We show that the strongest stereotypes in all models across all languages are that women are 'beautiful', 'empathetic' and 'neat' and men are 'leaders','strong, tough' and 'professional'
  arXiv  Detail & Related papers  (2025-06-04T11:58:18Z)
• Adapting Psycholinguistic Research for LLMs: Gender-inclusive Language in a Coreference Context [0.9831489366502298]
  This study examines whether Large Language Models interpret gender-inclusive language neutrally.<n>In English, LLMs generally maintain the antecedent's gender but exhibit underlying masculine bias.<n>In German, this bias is much stronger, overriding all tested gender-neutralization strategies.
  arXiv  Detail & Related papers  (2025-02-18T18:42:11Z)
• Mind the Gap! Choice Independence in Using Multilingual LLMs for Persuasive Co-Writing Tasks in Different Languages [51.96666324242191]
  We analyze whether user utilization of novel writing assistants in a charity advertisement writing task is affected by the AI's performance in a second language.<n>We quantify the extent to which these patterns translate into the persuasiveness of generated charity advertisements.
  arXiv  Detail & Related papers  (2025-02-13T17:49:30Z)
• The Dark Patterns of Personalized Persuasion in Large Language Models: Exposing Persuasive Linguistic Features for Big Five Personality Traits in LLMs Responses [0.0]
  We identify 13 linguistic features crucial for influencing personalities across different levels of the Big Five model of personality. Findings show that models use more anxiety-related words for neuroticism, increase achievement-related words for conscientiousness, and employ fewer cognitive processes words for openness to experience.
  arXiv  Detail & Related papers  (2024-11-08T23:02:59Z)
• Large Language Models Reflect the Ideology of their Creators [71.65505524599888]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language.<n>This paper shows that the ideological stance of an LLM appears to reflect the worldview of its creators.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• Measuring and Benchmarking Large Language Models' Capabilities to Generate Persuasive Language [41.052284715017606]
  We study the ability of Large Language Models (LLMs) to produce persuasive text.<n>As opposed to prior work which focuses on particular domains or types of persuasion, we conduct a general study across various domains.<n>We construct the new dataset Persuasive-Pairs of pairs of pairs of a short text and its rewrite by an LLM to amplify or diminish persuasive language.
  arXiv  Detail & Related papers  (2024-06-25T17:40:47Z)
• BabySLM: language-acquisition-friendly benchmark of self-supervised spoken language models [56.93604813379634]
  Self-supervised techniques for learning speech representations have been shown to develop linguistic competence from exposure to speech without the need for human labels. We propose a language-acquisition-friendly benchmark to probe spoken language models at the lexical and syntactic levels. We highlight two exciting challenges that need to be addressed for further progress: bridging the gap between text and speech and between clean speech and in-the-wild speech.
  arXiv  Detail & Related papers  (2023-06-02T12:54:38Z)
• Multilingual Contextual Affective Analysis of LGBT People Portrayals in Wikipedia [34.183132688084534]
  Specific lexical choices in narrative text reflect both the writer's attitudes towards people in the narrative and influence the audience's reactions. We show how word connotations differ across languages and cultures, highlighting the difficulty of generalizing existing English datasets. We then demonstrate the usefulness of our method by analyzing Wikipedia biography pages of members of the LGBT community across three languages.
  arXiv  Detail & Related papers  (2020-10-21T08:27:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.