Revising Context, Shifting Simulated Stance: Auditing LLM-Based Stance Simulation in Online Discussions

• URL: http://arxiv.org/abs/2606.06443v2
• Date: Mon, 08 Jun 2026 14:12:52 GMT
• Title: Revising Context, Shifting Simulated Stance: Auditing LLM-Based Stance Simulation in Online Discussions
• Abstract summary: Large language models are increasingly used to simulate social media users.<n>We study counterfactual context revision as a framework for auditing LLM-based stance simulation.<n>Results reveal effective and robust stance transitions in both text-only and multimodal strategies.
• Score: 62.99524765341952
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models are increasingly used to simulate social media users and infer how individuals may respond to online discussions. However, it remains unclear whether these simulations reflect precise user-specific beliefs or whether they are highly sensitive to semantically independent changes in conversational contexts. In this work, we study counterfactual context revision as a framework for auditing LLM-based stance simulation. Given an original online conversation, we first infer a target user's stance toward a specific topic. We then apply controlled revision strategies to the conversational context and simulate the user's stance again under the revised context. We compare text-only revision strategies with a multimodal one that incorporates meme-based context and evaluate two main effectiveness metrics, i.e., average directional stance shift and stance transition rate. The results reveal effective and robust stance transitions in both text-only and multimodal strategies across different polarization-preference mechanisms. Our study contributes an evaluation framework for understanding the context sensitivity of LLM-based stance simulation. More broadly, it highlights both the promise and risk of using LLMs to simulate online opinion dynamics.

Related papers

• SWIM: Student Writing Simulation via Proficiency-Conditioned Generation [54.69949380133243]
  This work explores if language models can realistically simulate student writing, and introduce SWIM.<n>We evaluate prompting, supervised fine-tuning, and reinforcement learning methods for writing simulation using automated essay scoring as a measure of profile alignment.<n>Our findings suggest that explicit supervision enables substantially stronger profile alignment than prompting alone, while authentic low-proficiency writing remains challenging to reproduce.
  arXiv  Detail & Related papers  (2026-09-02T23:10:51Z)
• Semantic Variability of Replies Across LLMs: Implications for Designing Conversation-Based Assessment [0.0]
  This study examines whether LLM-generated replies remain semantically consistent when the underlying LLM changes.<n>Results show that model choice and conversational context both affect response similarity and alignment with human replies.
  arXiv  Detail & Related papers  (2026-08-13T19:22:09Z)
• Contextualized Evaluation of Vision Language Models through Dynamic, Multi-turn Interactions [15.505991441542372]
  We propose a framework that recasts evaluation as a three-party interaction between an evaluatee model, an automated examiner, and a grader.<n>By navigating state-space transitions, CEDI deploys diverse strategies, from clarification requests to adversarial probes, to elicit performance evidence.<n>We show that hallucinations often accumulate over long contexts, through self-reinforcing dialogue history, and models are particularly vulnerable to questions requiring premise rejection or refusal.
  arXiv  Detail & Related papers  (2026-07-16T02:25:03Z)
• In-Context Model Predictive Generation: Open-Vocabulary Motion Synthesis from Language Models to Physics [109.99871326497333]
  In-Context Model Predictive Generation (ICMPG) is a framework that integrates language-model planning with inference-time physical feedback.<n>ICMPG reformulates motion synthesis as a Model Predictive Control (MPC)-like process with two modules.
  arXiv  Detail & Related papers  (2026-06-25T12:50:33Z)
• From Static Context to Calibrated Interactive RL: Mitigating Distribution Shift in Multi-turn Dialogue with Aligned Simulator [33.62561779855603]
  A long-standing goal of the research community is to develop highly interactive LLM-based dialogue agents.<n>Recent research focuses on optimizing policies based on fixed offline logs (Static Context RL) or using a prompt-based simulator (Interactive RL)<n>We show that both paradigms are fundamentally limited by context distribution shift.<n>We propose Calibrated Interactive RL, a unified framework that couples interactive RL with simulator alignment.
  arXiv  Detail & Related papers  (2026-05-26T00:25:12Z)
• Do LLM-Driven Agents Exhibit Engagement Mechanisms? Controlled Tests of Information Load, Descriptive Norms, and Popularity Cues [28.621781661498545]
  We evaluate what an LLM-driven simulation can credibly support using information engagement on social media as a test case.<n>In a Weibo-like environment, we manipulate information load and descriptive norms, while allowing popularity cues to evolve endogenously.<n>We ask whether simulated behavior changes in theoretically interpretable ways under these controlled variations, rather than merely producing plausible-looking traces.
  arXiv  Detail & Related papers  (2026-03-21T18:50:22Z)
• Simulation-in-the-Reasoning (SiR): A Conceptual Framework for Empirically Grounded AI in Autonomous Transportation [0.0]
  This paper introduces Simulation-in-the-Reasoning (SiR), a novel conceptual framework that embeds domain-specific simulators directly into the Large Language Models (LLMs) reasoning loop.<n>We argue that SiR represents a critical step towards trustworthy, empirically-validated AI for autonomous transportation systems.
  arXiv  Detail & Related papers  (2026-03-11T00:27:13Z)
• Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning [52.07170679746533]
  Large Language Models (LLMs) are increasingly used to simulate human users in interactive settings such as therapy, education, and social role-play.<n>We introduce a unified framework for evaluating and improving persona consistency in LLM-generated dialogue.<n>We define three automatic metrics: prompt-to-line consistency, line-to-line consistency, and Q&A consistency, that capture different types of persona drift and validate each against human annotations.
  arXiv  Detail & Related papers  (2025-10-31T19:40:41Z)
• DEBATE: A Large-Scale Benchmark for Role-Playing LLM Agents in Multi-Agent, Long-Form Debates [10.609797175227644]
  We introduce DEBATE, the first large-scale empirical benchmark to evaluate the authenticity of the interaction between multi-agent role-playing LLMs.<n>We systematically evaluate and identify critical discrepancies between simulated and authentic group dynamics.
  arXiv  Detail & Related papers  (2025-10-29T02:21:10Z)
• LANPO: Bootstrapping Language and Numerical Feedback for Reinforcement Learning in LLMs [73.27182315028021]
  LANPO is a framework that cleanly separates the roles of feedback: language guides exploration, while numerical rewards drive optimization.<n>Our work provides a robust method for integrating historical experiences into the LLM RL loop, creating more effective and data-efficient learning agents.
  arXiv  Detail & Related papers  (2025-10-18T15:51:19Z)
• Sample-Efficient Online Learning in LM Agents via Hindsight Trajectory Rewriting [92.57796055887995]
  We introduce ECHO, a prompting framework that adapts hindsight experience replay from reinforcement learning for language model agents.<n> ECHO generates optimized trajectories for alternative goals that could have been achieved during failed attempts.<n>We evaluate ECHO on stateful versions of XMiniGrid, a text-based navigation and planning benchmark, and PeopleJoinQA, a collaborative information-gathering enterprise simulation.
  arXiv  Detail & Related papers  (2025-10-11T18:11:09Z)
• A Personalized Conversational Benchmark: Towards Simulating Personalized Conversations [112.81207927088117]
  PersonaConvBench is a benchmark for evaluating personalized reasoning and generation in multi-turn conversations with large language models (LLMs)<n>We benchmark several commercial and open-source LLMs under a unified prompting setup and observe that incorporating personalized history yields substantial performance improvements.
  arXiv  Detail & Related papers  (2025-05-20T09:13:22Z)
• Dynamic benchmarking framework for LLM-based conversational data capture [0.0]
  This paper introduces a benchmarking framework to assess large language models (LLMs)<n>It integrates generative agent simulation to evaluate performance on key dimensions: information extraction, context awareness, and adaptive engagement.<n>Results show that adaptive strategies improve data extraction accuracy, especially when handling ambiguous responses.
  arXiv  Detail & Related papers  (2025-02-04T15:47:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.