When Roles Fail: Epistemic Constraints on Advocate Role Fidelity in LLM-Based Political Statement Analysis

• URL: http://arxiv.org/abs/2604.27228v1
• Date: Wed, 29 Apr 2026 21:50:28 GMT
• Title: When Roles Fail: Epistemic Constraints on Advocate Role Fidelity in LLM-Based Political Statement Analysis
• Abstract summary: This paper provides the first systematic empirical test of the assumption that models will reliably maintain their assigned roles.<n>We measure role fidelity across 60 political statements using four metrics: Role Drift Index (RDI), Drift Distance (EDD), Directional Drift Index (DDI), and Entropy-based Role Stability (ERS)<n>We identify two failure modes - the Epistemic Floor Effect (fact-check results create an absolute lower bound below which the legitimizing role cannot be maintained) and Role-Prior Conflict (training-time knowledge overrides role instructions for factually unambiguous statements)
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Democratic discourse analysis systems increasingly rely on multi-agent LLM pipelines in which distinct evaluator models are assigned adversarial roles to generate structured, multi-perspective assessments of political statements. A core assumption is that models will reliably maintain their assigned roles. This paper provides the first systematic empirical test of that assumption using the TRUST pipeline. We develop an epistemic stance classifier that identifies advocate roles from reasoning text without relying on surface vocabulary, and measure role fidelity across 60 political statements (30 English, 30 German) using four metrics: Role Drift Index (RDI), Expected Drift Distance (EDD), Directional Drift Index (DDI), and Entropy-based Role Stability (ERS). We identify two failure modes - the Epistemic Floor Effect (fact-check results create an absolute lower bound below which the legitimizing role cannot be maintained) and Role-Prior Conflict (training-time knowledge overrides role instructions for factually unambiguous statements) - as manifestations of a single mechanism: Epistemic Role Override (ERO). Model choice significantly affects role fidelity: Mistral Large outperforms Claude Sonnet by 28pp (67% vs. 39%) and exhibits a qualitatively different failure mode - role abandonment without polarity reversal - compared to Claude's active switch to the opposing stance. Role fidelity is language-robust. Fact-check provider choice is not universally neutral: Perplexity significantly reduces Claude's role fidelity on German statements (Delta = -15pp, p = 0.007) while leaving Mistral unaffected. These findings have direct implications for multi-agent LLM validation: a system validated without role fidelity measurement may systematically misrepresent the epistemic diversity it was designed to provide.

Related papers

• Inference-Time Steering for Cross-Lingual Factual Consistency in LLMs [51.56484100374058]
  Large Language Models (LLMs) demonstrate remarkable multilingual fluency, but their internal knowledge representations remain disproportionately biased toward high-resource languages.<n>We investigate whether these biases can be mitigated at inference time, forcing an English-prompted model to answer as if it were queried in target languages.<n>Experiments on Gemma 3 12B Instruct reveal persona prompting to be the strongest overall intervention, balancing efficacy, safety, and out-of-domain generalization.
  arXiv  Detail & Related papers  (2026-07-21T16:15:05Z)
• Do Modules Stay in Their Lane? Role Drift in Compound LLM Systems [0.66816887202191]
  We identify Role Drift, a failure mode in which modules deviating from their assigned roles remain invisible to system-level evaluation.<n>We propose Role Anchor, a regularizer that modulates how much each module deviates from its assigned role during end-to-end training.
  arXiv  Detail & Related papers  (2026-07-07T22:31:23Z)
• The Granularity Axis: A Micro-to-Macro Latent Direction for Social Roles in Language Models [61.32031248060941]
  We show that large language models encode social roles, from micro-level individual experience to macro-level organizational, institutional, or national reasoning.<n>Our findings suggest that social role granularity is not merely a stylistic surface feature, but a structured, ordered, and causally manipulable latent direction in language model behavior.
  arXiv  Detail & Related papers  (2026-05-07T13:08:19Z)
• Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
  Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
  arXiv  Detail & Related papers  (2026-04-21T15:05:58Z)
• IV Co-Scientist: Multi-Agent LLM Framework for Causal Instrumental Variable Discovery [61.15184885636171]
  In the presence of confounding between an endogenous variable and the outcome, instrumental variables (IVs) are used to isolate the causal effect of the endogenous variable.<n>We investigate whether large language models (LLMs) can aid in this task.<n>We introduce IV Co-Scientist, a multi-agent system that proposes, critiques, and refines IVs for a given treatment-outcome pair.
  arXiv  Detail & Related papers  (2026-02-08T12:28:29Z)
• Personalization Increases Affective Alignment but Has Role-Dependent Effects on Epistemic Independence in LLMs [2.1198879079315573]
  Large Language Models (LLMs) are prone to sycophantic behavior, uncritically conforming to user beliefs.<n>We present a rigorous evaluation of personalization's impact on LLM sycophancy across nine frontier models and five benchmark datasets.
  arXiv  Detail & Related papers  (2026-02-03T19:04:03Z)
• Improving LLM Reasoning through Interpretable Role-Playing Steering [33.25597755294326]
  Role-playing has emerged as an effective technique for enhancing the reasoning capabilities of large language models (LLMs)<n>We introduce Sparse Autoencoder Role-Playing Steering (SRPS), a novel framework that identifies and manipulates internal model features associated with role-playing behavior.<n>Our approach extracts latent representations from role-play prompts, selects the most relevant features based on activation patterns, and constructs a steering vector that can be injected into the model's residual stream with controllable intensity.
  arXiv  Detail & Related papers  (2025-06-09T00:31:17Z)
• Separator Injection Attack: Uncovering Dialogue Biases in Large Language Models Caused by Role Separators [18.00739443633401]
  Role separators are employed to distinguish between different participants in a conversation.<n>Misusing roles can lead to prompt injection attacks, which can easily misalign the model's behavior with the user's intentions.<n>We develop the Separators Injection Attack (SIA), a new orthometric attack based on role separators.
  arXiv  Detail & Related papers  (2025-04-08T05:20:56Z)
• MIRAGE: Multimodal Immersive Reasoning and Guided Exploration for Red-Team Jailbreak Attacks [85.3303135160762]
  MIRAGE is a novel framework that exploits narrative-driven context and role immersion to circumvent safety mechanisms in Multimodal Large Language Models.<n>It achieves state-of-the-art performance, improving attack success rates by up to 17.5% over the best baselines.<n>We demonstrate that role immersion and structured semantic reconstruction can activate inherent model biases, facilitating the model's spontaneous violation of ethical safeguards.
  arXiv  Detail & Related papers  (2025-03-24T20:38:42Z)
• Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistencies [66.30619782227173]
  Large language models (LLMs) can produce erroneous responses that sound fluent and convincing.<n>We identify several features of LLM responses that shape users' reliance.<n>We find that explanations increase reliance on both correct and incorrect responses.<n>We observe less reliance on incorrect responses when sources are provided or when explanations exhibit inconsistencies.
  arXiv  Detail & Related papers  (2025-02-12T16:35:41Z)
• Role-Play Paradox in Large Language Models: Reasoning Performance Gains and Ethical Dilemmas [7.677029165197536]
  Role-play in large language models (LLMs) enhances their ability to generate contextually relevant and high-quality responses.<n>We demonstrate that autotuning, a method used to auto-select models' roles based on the question, can lead to the generation of harmful outputs.
  arXiv  Detail & Related papers  (2024-09-21T02:09:13Z)
• When Prompting Fails to Sway: Inertia in Moral and Value Judgments of Large Language Models [4.906478894661688]
  Large Language Models (LLMs) exhibit non-deterministic behavior, and prompting has emerged as a primary method for steering their outputs toward desired directions.<n>One popular strategy involves assigning a specific "persona" to the model to induce more varied and context-sensitive responses, akin to the diversity found in human perspectives.<n>Our experiments demonstrate that LLMs maintain consistent value orientations.<n>In particular, we observe a persistent inertia in their responses, where certain moral and value dimensions, especially harm avoidance and fairness, remain distinctly skewed in one direction despite varied persona settings.
  arXiv  Detail & Related papers  (2024-08-16T23:24:10Z)
• Evaluating Interventional Reasoning Capabilities of Large Language Models [58.52919374786108]
  Large language models (LLMs) are used to automate decision-making tasks.<n>In this paper, we evaluate whether LLMs can accurately update their knowledge of a data-generating process in response to an intervention.<n>We create benchmarks that span diverse causal graphs (e.g., confounding, mediation) and variable types.<n>These benchmarks allow us to isolate the ability of LLMs to accurately predict changes resulting from their ability to memorize facts or find other shortcuts.
  arXiv  Detail & Related papers  (2024-04-08T14:15:56Z)
• Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment [62.898963074989766]
  We introduce Ditto, a self-alignment method for role-play. This method creates a role-play training set comprising 4,000 characters, surpassing the scale of currently available datasets by tenfold. We present the first comprehensive cross-supervision alignment experiment in the role-play domain.
  arXiv  Detail & Related papers  (2024-01-23T03:56:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.