Mixture of Debaters: Learn to Debate at Architectural Level in Multi-Agent Reasoning

• URL: http://arxiv.org/abs/2606.29425v1
• Date: Sun, 28 Jun 2026 14:40:01 GMT
• Title: Mixture of Debaters: Learn to Debate at Architectural Level in Multi-Agent Reasoning
• Abstract summary: Mixture of Debaters (MoD) is a unified framework that enables dynamic self-debate within a single model.<n>We address three key challenges in adapting MoE for dialectical reasoning.<n>Experiments on multimodal benchmarks demonstrate that MoD outperforms both single-model baselines and conventional multi-agent systems.
• Score: 16.45665127823716
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing multi-agent debate frameworks suffer from two critical limitations: they rely on static architectures where agent roles and coordination patterns are fixed at design time, and they require instantiating multiple model copies, incurring substantial computational overhead. We propose Mixture of Debaters (MoD), a unified framework that enables dynamic self-debate within a single model by leveraging the Mixture-of-Experts paradigm. We address three key challenges in adapting MoE for dialectical reasoning: (1) dual-routing that decouples role allocation from process flow, dynamically determining when to debate versus when to synthesize; (2) momentum switching that smooths token-level routing with local context, reducing expert-switch jitter; and (3) unified self-debate that encapsulates diverse debating personas into lightweight expert modules, eliminating inter-agent communication while preserving behavioral diversity. Extensive experiments on multimodal benchmarks demonstrate that MoD outperforms both single-model baselines and conventional multi-agent systems, achieving superior accuracy with 3.7x lower latency and 87% reduction in token consumption.The source code can be accessed at https://github.com/YongLD/MoD.

Related papers

• Switch-Reasoner: Learn When to Think in Multitask Mixtures via Reinforcement Learning [72.45536737928015]
  We propose Switch-Reasoner, a framework that learns to adaptively select reasoning modes for MLLMs.<n>It treats thinking as a virtual tool invocation and allows the model to either answer directly or invoke explicit reasoning before answering.<n>Experiments on 11 multimodal tasks show that Switch-Reasoner reduces unnecessary reasoning while maintaining strong performance.
  arXiv  Detail & Related papers  (2026-07-09T15:04:05Z)
• Latent Agents: A Post-Training Procedure for Internalized Multi-Agent Debate [16.68699018207298]
  Multi-agent debate has been shown to improve reasoning in large language models (LLMs)<n>We develop a framework that distills multi-agent debate into a single LLM through a two-stage fine-tuning pipeline.<n>Our internalized models match or exceed explicit multi-agent debate performance using up to 93% fewer tokens.
  arXiv  Detail & Related papers  (2026-04-27T18:06:03Z)
• BiManiBench: A Hierarchical Benchmark for Evaluating Bimanual Coordination of Multimodal Large Language Models [30.38546017617527]
  We introduce BiManiBench, a benchmark evaluating Multi Large Language Models (MLLMs) across three tiers.<n>Our framework isolates unique bimanual challenges, such as arm reachability and kinematic constraints.<n>Analysis of over 30 state-of-the-art models reveals that despite high-level reasoning proficiency, MLLMs struggle with dual-arm spatial grounding and control.
  arXiv  Detail & Related papers  (2026-02-09T08:47:14Z)
• AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agent [57.10083973844841]
  AgentArk is a novel framework to distill multi-agent dynamics into the weights of a single model.<n>We investigate three hierarchical distillation strategies across various models, tasks, scaling, and scenarios.<n>By shifting the burden of computation from inference to training, the distilled models preserve the efficiency of one agent while exhibiting strong reasoning and self-correction performance of multiple agents.
  arXiv  Detail & Related papers  (2026-02-03T19:18:28Z)
• TS-Debate: Multimodal Collaborative Debate for Zero-Shot Time Series Reasoning [44.59910717749994]
  We present TS-Debate, a modality-specialized, collaborative multi-agent debate framework for zero-shot time series reasoning.<n>TS-Debate assigns dedicated expert agents to textual context, visual patterns, and numerical signals, preceded by explicit domain knowledge elicitation.<n>Reviewer agents evaluate agent claims using a verification-conflict-calibration mechanism, supported by lightweight code execution and numerical lookup.
  arXiv  Detail & Related papers  (2026-01-27T03:29:22Z)
• DynaDebate: Breaking Homogeneity in Multi-Agent Debate with Dynamic Path Generation [47.62978918069135]
  We introduce Dynamic Multi-Agent Debate (DynaDebate), which enhances the effectiveness of multi-agent debate through three key mechanisms.<n>Extensive experiments demonstrate that DynaDebate achieves superior performance across various benchmarks, surpassing existing state-of-the-art MAD methods.
  arXiv  Detail & Related papers  (2026-01-09T12:01:33Z)
• Single LLM Debate, MoLaCE: Mixture of Latent Concept Experts Against Confirmation Bias [24.182306712604966]
  Large language models (LLMs) are highly vulnerable to input confirmation bias.<n>MoLaCE is a lightweight inference-time framework that addresses confirmation bias by mixing experts instantiated as different activation strengths.<n>We empirically show that it consistently reduces confirmation bias, improves robustness, and surpasses multi-agent debate.
  arXiv  Detail & Related papers  (2025-12-29T14:52:34Z)
• AMUSE: Audio-Visual Benchmark and Alignment Framework for Agentic Multi-Speaker Understanding [73.05946667683259]
  Recent large language models (MLLMs) show strong perception but struggle in multi-speaker, dialogue-centric settings.<n>We introduce AMUSE, a benchmark designed around tasks that are inherently agentic.<n>We propose RAFT, a data-efficient agentic alignment framework that integrates reward optimization with intrinsic multimodal self-evaluation.
  arXiv  Detail & Related papers  (2025-12-18T07:01:47Z)
• NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matching [64.10695425442164]
  We introduce NExT-OMNI, an open-source omnimodal foundation model that achieves unified modeling through discrete flow paradigms.<n>Trained on large-scale interleaved text, image, video, and audio data, NExT-OMNI delivers competitive performance on multimodal generation and understanding benchmarks.<n>To advance further research, we release training details, data protocols, and open-source both the code and model checkpoints.
  arXiv  Detail & Related papers  (2025-10-15T16:25:18Z)
• MADIAVE: Multi-Agent Debate for Implicit Attribute Value Extraction [52.89860691282002]
  Implicit Attribute Value Extraction (AVE) is essential for accurately representing products in e-commerce.<n>Despite advances in multimodal large language models (MLLMs), implicit AVE remains challenging due to the complexity of multidimensional data.<n>We introduce textscmodelname, a multi-agent debate framework that employs multiple MLLM agents to iteratively refine inferences.
  arXiv  Detail & Related papers  (2025-10-07T06:27:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.