Generative Exaggeration in LLM Social Agents: Consistency, Bias, and Toxicity

• URL: http://arxiv.org/abs/2507.00657v1
• Date: Tue, 01 Jul 2025 10:54:51 GMT
• Title: Generative Exaggeration in LLM Social Agents: Consistency, Bias, and Toxicity
• Abstract summary: We investigate how Large Language Models (LLMs) behave when simulating political discourse on social media.<n>We construct LLM agents based on 1,186 real users, prompting them to reply to politically salient tweets under controlled conditions.<n>We find that richer contextualization improves internal consistency but also amplifies polarization, stylized signals, and harmful language.
• Score: 2.3997896447030653
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We investigate how Large Language Models (LLMs) behave when simulating political discourse on social media. Leveraging 21 million interactions on X during the 2024 U.S. presidential election, we construct LLM agents based on 1,186 real users, prompting them to reply to politically salient tweets under controlled conditions. Agents are initialized either with minimal ideological cues (Zero Shot) or recent tweet history (Few Shot), allowing one-to-one comparisons with human replies. We evaluate three model families (Gemini, Mistral, and DeepSeek) across linguistic style, ideological consistency, and toxicity. We find that richer contextualization improves internal consistency but also amplifies polarization, stylized signals, and harmful language. We observe an emergent distortion that we call "generation exaggeration": a systematic amplification of salient traits beyond empirical baselines. Our analysis shows that LLMs do not emulate users, they reconstruct them. Their outputs, indeed, reflect internal optimization dynamics more than observed behavior, introducing structural biases that compromise their reliability as social proxies. This challenges their use in content moderation, deliberative simulations, and policy modeling.

Related papers

• How Identity and Opinion Shape Political Sycophancy in LLMs [17.0131514909503]
  We introduce a framework that disentangles two distinct triggers of political sycophancy: opinion and identity.<n>Using 450 manually-checked political dilemmas as controlled probes, we evaluate 13 instruction-tuned Large Language Models.<n>Our results suggest that LLM political stance is interactively and steerably vulnerable rather than being a fixed trait.
  arXiv  Detail & Related papers  (2026-08-29T11:12:03Z)
• Verbalizing LLMs' assumptions to explain and control sycophancy [62.927670321859495]
  LLMs can be socially sycophantic, affirming users when they ask questions like "am I in the wrong?" rather than providing genuine assessment.<n>We present Verbalized Assumptions, a framework for eliciting these assumptions from LLMs.<n>Our work contributes a new understanding of assumptions as a mechanism for sycophancy.
  arXiv  Detail & Related papers  (2026-04-03T14:15:43Z)
• Do Large Language Models Adapt to Language Variation across Socioeconomic Status? [29.1246345717672]
  Humans adjust their linguistic style to the audience they are addressing.<n>As these models increasingly mediate human-to-human communication, their failure to adapt to diverse styles can perpetuate stereotypes and marginalize communities.<n>We study the extent to which LLMs integrate into social media communication across different socioeconomic status (SES) communities.
  arXiv  Detail & Related papers  (2026-02-12T13:36:38Z)
• Us-vs-Them bias in Large Language Models [0.569978892646475]
  We find consistent ingroup-positive and outgroup-negative associations across foundational large language models.<n>For personas examined, conservative personas exhibit greater outgroup hostility, whereas liberal personas display stronger ingroup solidarity.
  arXiv  Detail & Related papers  (2025-12-03T07:11:22Z)
• DEBATE: A Large-Scale Benchmark for Role-Playing LLM Agents in Multi-Agent, Long-Form Debates [10.609797175227644]
  We introduce DEBATE, the first large-scale empirical benchmark to evaluate the authenticity of the interaction between multi-agent role-playing LLMs.<n>We systematically evaluate and identify critical discrepancies between simulated and authentic group dynamics.
  arXiv  Detail & Related papers  (2025-10-29T02:21:10Z)
• Social Simulations with Large Language Model Risk Utopian Illusion [61.358959720048354]
  We introduce a systematic framework for analyzing large language models' behavior in social simulation.<n>Our approach simulates multi-agent interactions through chatroom-style conversations and analyzes them across five linguistic dimensions.<n>Our findings reveal that LLMs do not faithfully reproduce genuine human behavior but instead reflect overly idealized versions of it.
  arXiv  Detail & Related papers  (2025-10-24T06:08:41Z)
• Spiral of Silence in Large Language Model Agents [44.98734791415891]
  The Spiral of Silence (SoS) theory holds that individuals with minority views often refrain from speaking out for fear of social isolation.<n>This raises a central question: can SoS-like dynamics emerge from purely statistical language generation in large language models?<n>We consider four controlled conditions that systematically vary the availability of 'History' and 'Persona' signals.
  arXiv  Detail & Related papers  (2025-09-28T08:59:54Z)
• Revisiting LLM Value Probing Strategies: Are They Robust and Expressive? [81.49470136653665]
  We evaluate the robustness and expressiveness of value representations across three widely used probing strategies.<n>We show that the demographic context has little effect on the free-text generation, and the models' values only weakly correlate with their preference for value-based actions.
  arXiv  Detail & Related papers  (2025-07-17T18:56:41Z)
• Machine Bullshit: Characterizing the Emergent Disregard for Truth in Large Language Models [57.834711966432685]
  Bullshit, as conceptualized by philosopher Harry Frankfurt, refers to statements made without regard to their truth value.<n>We introduce the Bullshit Index, a novel metric quantifying large language model's indifference to truth.<n>We observe prevalent machine bullshit in political contexts, with weasel words as the dominant strategy.
  arXiv  Detail & Related papers  (2025-07-10T07:11:57Z)
• Too Human to Model:The Uncanny Valley of LLMs in Social Simulation -- When Generative Language Agents Misalign with Modelling Principles [0.0]
  Large language models (LLMs) have been increasingly used to build agents in social simulation.<n>We argue that LLM agents are too expressive, detailed and intractable to be consistent with the abstraction, simplification, and interpretability typically demanded by modelling.
  arXiv  Detail & Related papers  (2025-07-08T18:02:36Z)
• Passing the Turing Test in Political Discourse: Fine-Tuning LLMs to Mimic Polarized Social Media Comments [0.0]
  This study explores the extent to which fine-tuned large language models (LLMs) can replicate and amplify polarizing discourse.<n>Using a curated dataset of politically charged discussions extracted from Reddit, we fine-tune an open-source LLM to produce context-aware and ideologically aligned responses.<n>The results indicate that, when trained on partisan data, LLMs are capable of producing highly plausible and provocative comments, often indistinguishable from those written by humans.
  arXiv  Detail & Related papers  (2025-06-17T15:41:26Z)
• Through the LLM Looking Glass: A Socratic Probing of Donkeys, Elephants, and Markets [42.55423041662188]
  The study aims to directly measure the models' biases rather than relying on external interpretations.<n>Our results reveal a consistent preference of Democratic over Republican positions across all models.<n>In economic topics, biases vary among Western LLMs, while those developed in China lean more strongly toward socialism.
  arXiv  Detail & Related papers  (2025-03-20T19:40:40Z)
• Can LLMs Simulate Social Media Engagement? A Study on Action-Guided Response Generation [51.44040615856536]
  This paper analyzes large language models' ability to simulate social media engagement through action guided response generation.<n>We benchmark GPT-4o-mini, O1-mini, and DeepSeek-R1 in social media engagement simulation regarding a major societal event.
  arXiv  Detail & Related papers  (2025-02-17T17:43:08Z)
• Incivility and Rigidity: The Risks of Fine-Tuning LLMs for Political Argumentation [11.255011967393838]
  Incivility prevalent on platforms like Twitter (now X) and Reddit poses a challenge for developing AI systems.<n>In this study, we report experiments with GPT-3.5 Turbo, fine-tuned on two contrasting datasets of political discussions.<n>We show that Reddit-finetuned models produce safer but rhetorically rigid arguments, while cross-platform fine-tuning amplifies toxicity.
  arXiv  Detail & Related papers  (2024-11-25T15:28:11Z)
• Large Language Models Reflect the Ideology of their Creators [71.65505524599888]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language.<n>This paper shows that the ideological stance of an LLM appears to reflect the worldview of its creators.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactions [25.809599403713506]
  Large Language Models (LLMs) are increasingly being employed in numerous studies to simulate societies and execute diverse social tasks. LLMs are susceptible to societal biases due to their exposure to human-generated data. This study investigates the presence of implicit gender biases in multi-agent LLM interactions and proposes two strategies to mitigate these biases.
  arXiv  Detail & Related papers  (2024-10-03T15:28:05Z)
• White Men Lead, Black Women Help? Benchmarking and Mitigating Language Agency Social Biases in LLMs [58.27353205269664]
  Social biases can manifest in language agency in Large Language Model (LLM)-generated content.<n>We introduce the Language Agency Bias Evaluation benchmark, which comprehensively evaluates biases in LLMs.<n>Using LABE, we unveil language agency social biases in 3 recent LLMs: ChatGPT, Llama3, and Mistral.
  arXiv  Detail & Related papers  (2024-04-16T12:27:54Z)
• Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement [75.7148545929689]
  Large language models (LLMs) improve their performance through self-feedback on certain tasks while degrade on others. We formally define LLM's self-bias - the tendency to favor its own generation. We analyze six LLMs on translation, constrained text generation, and mathematical reasoning tasks.
  arXiv  Detail & Related papers  (2024-02-18T03:10:39Z)
• Simulating Opinion Dynamics with Networks of LLM-based Agents [7.697132934635411]
  We propose a new approach to simulating opinion dynamics based on populations of Large Language Models (LLMs) Our findings reveal a strong inherent bias in LLM agents towards producing accurate information, leading simulated agents to consensus in line with scientific reality. After inducing confirmation bias through prompt engineering, however, we observed opinion fragmentation in line with existing agent-based modeling and opinion dynamics research.
  arXiv  Detail & Related papers  (2023-11-16T07:01:48Z)
• Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis [86.49858739347412]
  Large Language Models (LLMs) have sparked intense debate regarding the prevalence of bias in these models and its mitigation. We propose a prompt-based method for the extraction of confounding and mediating attributes which contribute to the decision process. We find that the observed disparate treatment can at least in part be attributed to confounding and mitigating attributes and model misalignment.
  arXiv  Detail & Related papers  (2023-11-15T00:02:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.