Hear Both Sides: Efficient Multi-Agent Debate via Diversity-Aware Message Retention

• URL: http://arxiv.org/abs/2603.20640v1
• Date: Sat, 21 Mar 2026 04:27:47 GMT
• Title: Hear Both Sides: Efficient Multi-Agent Debate via Diversity-Aware Message Retention
• Abstract summary: Multi-Agent Debate has emerged as a promising framework for improving the reasoning quality of large language models.<n>Current approaches rely on uncertainty estimation to filter low-confidence responses before broadcasting.<n>We propose Diversity-Aware Retention (DAR), a lightweight debate framework that selects the subset of agent responses that maximally disagree with each other.
• Score: 31.583055847273826
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-Agent Debate has emerged as a promising framework for improving the reasoning quality of large language models through iterative inter-agent communication. However, broadcasting all agent messages at every round introduces noise and redundancy that can degrade debate quality and waste computational resources. Current approaches rely on uncertainty estimation to filter low-confidence responses before broadcasting, but this approach is unreliable due to miscalibrated confidence scores and sensitivity to threshold selection. To address this, we propose Diversity-Aware Retention (DAR), a lightweight debate framework that, at each debate round, selects the subset of agent responses that maximally disagree with each other and with the majority vote before broadcasting. Through an explicit index-based retention mechanism, DAR preserves the original messages without modification, ensuring that retained disagreements remain authentic. Experiments on diverse reasoning and question answering benchmarks demonstrate that our selective message propagation consistently improves debate performance, particularly as the number of agents scales, where noise accumulation is most severe. Our results highlight that what agents hear is as important as what agents say in multi-agent reasoning systems.

Related papers

• Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation [6.5331167606521205]
  Multi-agent debate (MAD) improves the reasoning capabilities of large language models by having multiple agents iteratively refine their responses through discussion.<n>MAD suffers from a critical vulnerability known as shared misconception: when a majority of agents converge on an incorrect answer, the debate process tends to amplify rather than correct the error.<n>Existing methods primarily address peer skew but leave the agents' inherently biased concept priors unaddressed.<n>We propose R$2$-MAD, a framework that equips agents with an experience memory accumulated from past debates.
  arXiv  Detail & Related papers  (2026-09-03T10:05:01Z)
• Dynamic Trust-Aware Sparse Communication Topology for LLM-Based Multi-Agent Consensus [8.002017617206407]
  This paper proposes DySCo, a dynamic trust-aware sparse consensus mechanism.<n>It estimates the value of communication edges based on agent reliability, answer divergence, and task relevance, and selects a small number of high-value edges for message exchange.<n>It then aggregates the answers of different agents through dynamic trust weights and terminates the discussion early once consensus stabilizes.
  arXiv  Detail & Related papers  (2026-06-01T07:42:33Z)
• Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
  Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
  arXiv  Detail & Related papers  (2026-04-21T15:05:58Z)
• Debate to Align: Reliable Entity Alignment through Two-Stage Multi-Agent Debate [13.025981335381637]
  We propose AgentEA, a reliable EA framework based on multi-agent debate.<n>AgentEA improves embedding quality through entity representation preference optimization.<n>It then introduces a two-stage multi-role debate mechanism consisting of lightweight debate verification and deep debate alignment.
  arXiv  Detail & Related papers  (2026-04-15T06:54:27Z)
• AffectAgent: Collaborative Multi-Agent Reasoning for Retrieval-Augmented Multimodal Emotion Recognition [62.16431420189863]
  LLM-based multimodal emotion recognition relies on static parametric memory and often hallucinates when interpreting nuanced affective states.<n>We introduce AffectAgent, an affect-oriented multi-agent retrieval-augmented generation framework.<n>AffectAgent comprises three jointly optimized agents, namely a query planner, an evidence filter, and an emotion generator.
  arXiv  Detail & Related papers  (2026-04-14T13:49:19Z)
• EMS: Multi-Agent Voting via Efficient Majority-then-Stopping [62.95000188335248]
  We propose an Efficient Majority-then-Stopping (EMS) to improve reasoning efficiency.<n>EMS prioritizes agents based on task-aware reliability and terminates the reasoning pipeline the moment a majority is achieved.<n>EMS consistently reduces the average number of invoked agents by 32%.
  arXiv  Detail & Related papers  (2026-04-03T08:29:50Z)
• Balancing Faithfulness and Performance in Reasoning via Multi-Listener Soft Execution [79.98699884805636]
  Reasoning Execution by Multiple Listeners (REMUL) is a multi-party reinforcement learning approach.<n>REMUL builds on the hypothesis that reasoning traces which other parties can follow will be more faithful.<n>Speakers are rewarded for producing reasoning that is clear to listeners.
  arXiv  Detail & Related papers  (2026-02-18T02:55:55Z)
• DynaDebate: Breaking Homogeneity in Multi-Agent Debate with Dynamic Path Generation [47.62978918069135]
  We introduce Dynamic Multi-Agent Debate (DynaDebate), which enhances the effectiveness of multi-agent debate through three key mechanisms.<n>Extensive experiments demonstrate that DynaDebate achieves superior performance across various benchmarks, surpassing existing state-of-the-art MAD methods.
  arXiv  Detail & Related papers  (2026-01-09T12:01:33Z)
• OPTAGENT: Optimizing Multi-Agent LLM Interactions Through Verbal Reinforcement Learning for Enhanced Reasoning [14.105640933123325]
  Large Language Models (LLMs) have shown remarkable reasoning capabilities in mathematical and scientific tasks.<n>To enhance complex reasoning, multi-agent systems have been proposed to harness the collective intelligence of LLM agents.<n>We propose $ours$, a multi-agent verbal reinforcement learning algorithm that dynamically constructs and refines multi-agent collaboration structures.
  arXiv  Detail & Related papers  (2025-10-20T19:07:51Z)
• MADIAVE: Multi-Agent Debate for Implicit Attribute Value Extraction [52.89860691282002]
  Implicit Attribute Value Extraction (AVE) is essential for accurately representing products in e-commerce.<n>Despite advances in multimodal large language models (MLLMs), implicit AVE remains challenging due to the complexity of multidimensional data.<n>We introduce textscmodelname, a multi-agent debate framework that employs multiple MLLM agents to iteratively refine inferences.
  arXiv  Detail & Related papers  (2025-10-07T06:27:42Z)
• Debate Only When Necessary: Adaptive Multiagent Collaboration for Efficient LLM Reasoning [8.800516398660069]
  Multiagent collaboration has emerged as a promising framework for enhancing the reasoning capabilities of large language models (LLMs)<n>We propose Debate Only When Necessary (DOWN), an adaptive multiagent debate framework that selectively activates debate based on the confidence score of the agent's initial response.<n>Down improves efficiency by up to six times while preserving or even outperforming the performance of existing methods.
  arXiv  Detail & Related papers  (2025-04-07T13:17:52Z)
• DebUnc: Improving Large Language Model Agent Communication With Uncertainty Metrics [52.242449026151846]
  Multi-agent debates have been introduced to improve the accuracy of Large Language Models (LLMs)<n>We propose DebUnc, a debate framework that uses uncertainty metrics to assess agent confidence.
  arXiv  Detail & Related papers  (2024-07-08T22:15:01Z)
• ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs [61.07130026622437]
  Large Language Models (LLMs) still struggle with natural language reasoning tasks. Motivated by the society of minds, we propose ReConcile. A multi-model multi-agent framework designed as a round table conference among diverse LLM agents.
  arXiv  Detail & Related papers  (2023-09-22T17:12:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.