Belief in Authority: Impact of Authority in Multi-Agent Evaluation Framework

• URL: http://arxiv.org/abs/2601.04790v1
• Date: Thu, 08 Jan 2026 10:13:56 GMT
• Title: Belief in Authority: Impact of Authority in Multi-Agent Evaluation Framework
• Abstract summary: We present the first systematic analysis of role-based authority bias in free-form multi-agent evaluation using ChatEval.<n>Applying French and Raven's power-based theory, we classify authoritative roles into legitimate, referent, and expert types.<n>Experiments with GPT-4o and DeepSeek R1 reveal that Expert and Referent power roles exert stronger influence than Legitimate power roles.
• Score: 3.6812160259693094
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-agent systems utilizing large language models often assign authoritative roles to improve performance, yet the impact of authority bias on agent interactions remains underexplored. We present the first systematic analysis of role-based authority bias in free-form multi-agent evaluation using ChatEval. Applying French and Raven's power-based theory, we classify authoritative roles into legitimate, referent, and expert types and analyze their influence across 12-turn conversations. Experiments with GPT-4o and DeepSeek R1 reveal that Expert and Referent power roles exert stronger influence than Legitimate power roles. Crucially, authority bias emerges not through active conformity by general agents, but through authoritative roles consistently maintaining their positions while general agents demonstrate flexibility. Furthermore, authority influence requires clear position statements, as neutral responses fail to generate bias. These findings provide key insights for designing multi-agent frameworks with asymmetric interaction patterns.

Related papers

• Examining Agents' Bias Amplification versus Suppression in Multi-Agent Systems [5.210390383114933]
  We show that agents endowed with bias can substantially affect system-wide fairness.<n>When agents are exposed to bias uniformly, the system-wide bias elevates, even exceeding the additive sum of the individual agents' biases.
  arXiv  Detail & Related papers  (2026-05-27T07:53:29Z)
• Multi-Agent Systems are Mixtures of Experts: Who Becomes an Influencer? [31.047683581225087]
  We show that Friedkin-Johnsen (FJ) parameters are input-dependent, turning multi-agent deliberation into a mixture of experts.<n>We analyze how influence is established through observable proxies: agents' self-assessed confidence, their perceived confidence, and initial alignment with other agents' views.
  arXiv  Detail & Related papers  (2026-05-25T15:10:48Z)
• Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
  Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
  arXiv  Detail & Related papers  (2026-04-21T15:05:58Z)
• Making Bias Non-Predictive: Training Robust LLM Judges via Reinforcement Learning [91.8584139564909]
  Large language models (LLMs) increasingly serve as automated judges, yet they remain susceptible to cognitive biases.<n>We propose Epistemic Independence Training (EIT), a reinforcement learning framework grounded in a key principle.<n>EIT operationalizes this through a balanced conflict strategy where bias signals are equally likely to support correct and incorrect answers.
  arXiv  Detail & Related papers  (2026-02-02T01:43:48Z)
• The Why Behind the Action: Unveiling Internal Drivers via Agentic Attribution [63.61358761489141]
  Large Language Model (LLM)-based agents are widely used in real-world applications such as customer service, web navigation, and software engineering.<n>We propose a novel framework for textbfgeneral agentic attribution, designed to identify the internal factors driving agent actions regardless of the task outcome.<n>We validate our framework across a diverse suite of agentic scenarios, including standard tool use and subtle reliability risks like memory-induced bias.
  arXiv  Detail & Related papers  (2026-01-21T15:22:21Z)
• From Single to Societal: Analyzing Persona-Induced Bias in Multi-Agent Interactions [19.313710831511067]
  Large Language Model (LLM)-based multi-agent systems are increasingly used to simulate human interactions and solve collaborative tasks.<n>Do personas introduce biases into multi-agent interactions?<n>This paper presents a systematic investigation into persona-induced biases in multi-agent interactions.
  arXiv  Detail & Related papers  (2025-11-14T18:19:28Z)
• Disagreements in Reasoning: How a Model's Thinking Process Dictates Persuasion in Multi-Agent Systems [49.69773210844221]
  This paper challenges the prevailing hypothesis that persuasive efficacy is primarily a function of model scale.<n>Through a series of multi-agent persuasion experiments, we uncover a fundamental trade-off we term the Persuasion Duality.<n>Our findings reveal that the reasoning process in LRMs exhibits significantly greater resistance to persuasion, maintaining their initial beliefs more robustly.
  arXiv  Detail & Related papers  (2025-09-25T12:03:10Z)
• Helpful Agent Meets Deceptive Judge: Understanding Vulnerabilities in Agentic Workflows [41.97051158610974]
  We present a systematic analysis of agentic robustness under deceptive or misleading feedback.<n>We reveal that even strongest agents are vulnerable to persuasive yet flawed critiques.<n>Our findings highlight fundamental vulnerabilities in feedback-based robustness and offer guidance for building more robust agentic systems.
  arXiv  Detail & Related papers  (2025-06-03T19:26:23Z)
• An Adversary-Resistant Multi-Agent LLM System via Credibility Scoring [8.779871128906787]
  We introduce a general and adversary-resistant multi-agent LLM framework based on credibility scoring.<n>Our system associates a credibility score that is used when aggregating the team outputs.
  arXiv  Detail & Related papers  (2025-05-30T05:57:37Z)
• Judging with Many Minds: Do More Perspectives Mean Less Prejudice? On Bias Amplifications and Resistance in Multi-Agent Based LLM-as-Judge [70.89799989428367]
  We conduct a systematic analysis of four diverse bias types: position bias, verbosity bias, chain-of-thought bias, and bandwagon bias.<n>We evaluate these biases across two widely adopted multi-agent LLM-as-Judge frameworks: Multi-Agent-Debate and LLM-as-Meta-Judge.
  arXiv  Detail & Related papers  (2025-05-26T03:56:41Z)
• Interactional Fairness in LLM Multi-Agent Systems: An Evaluation Framework [0.0]
  We introduce a novel framework for evaluating Interactional fairness encompassing Interpersonal fairness (IF) and Informational fairness (InfF) in multi-agent systems.<n>We validate our framework through a pilot study using controlled simulations of a resource negotiation task.<n>Results show that tone and justification quality significantly affect acceptance decisions even when objective outcomes are held constant.
  arXiv  Detail & Related papers  (2025-05-17T13:24:13Z)
• Media and responsible AI governance: a game-theoretic and LLM analysis [61.132523071109354]
  This paper investigates the interplay between AI developers, regulators, users, and the media in fostering trustworthy AI systems.<n>Using evolutionary game theory and large language models (LLMs), we model the strategic interactions among these actors under different regulatory regimes.
  arXiv  Detail & Related papers  (2025-03-12T21:39:38Z)
• Problem-Solving in Language Model Networks [44.99833362998488]
  This work extends the concept of multi-agent debate to more general network topologies. It measures the question-answering accuracy, influence, consensus, and the effects of bias on the collective.
  arXiv  Detail & Related papers  (2024-06-18T07:59:14Z)
• SocialBench: Sociality Evaluation of Role-Playing Conversational Agents [85.6641890712617]
  Large language models (LLMs) have advanced the development of various AI conversational agents. SocialBench is the first benchmark designed to evaluate the sociality of role-playing conversational agents at both individual and group levels. We find that agents excelling in individual level does not imply their proficiency in group level.
  arXiv  Detail & Related papers  (2024-03-20T15:38:36Z)
• Learning to Incentivize Other Learning Agents [73.03133692589532]
  We show how to equip RL agents with the ability to give rewards directly to other agents, using a learned incentive function. Such agents significantly outperform standard RL and opponent-shaping agents in challenging general-sum Markov games. Our work points toward more opportunities and challenges along the path to ensure the common good in a multi-agent future.
  arXiv  Detail & Related papers  (2020-06-10T20:12:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.