L-MAD: A Systematic Evaluation of Multi-Agent Debate Structures in Legal Reasoning

• URL: http://arxiv.org/abs/2607.09099v1
• Date: Fri, 10 Jul 2026 05:08:02 GMT
• Title: L-MAD: A Systematic Evaluation of Multi-Agent Debate Structures in Legal Reasoning
• Abstract summary: We introduce the Legal Multi-Agent Debate (L-MAD) framework to evaluate different debate structures and aggregation methods within Legal Textual Entailment.<n>By assigning distinct expert personas to multiple agents, L-MAD improves upon strong single-agent baselines by up to 8%.
• Score: 14.460920090407017
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While multi-agent debate (MAD) frameworks have shown significant potential in general reasoning, their effectiveness in highly structured, knowledge-heavy legal domains remains under-explored. In this work, we introduce the Legal Multi-Agent Debate (L-MAD) framework to systematically evaluate different debate structures and aggregation methods within Legal Textual Entailment. By assigning distinct expert personas to multiple agents, L-MAD improves upon strong single-agent baselines by up to 8\%. Furthermore, analyzing how debate scales reveals a clear trade-off: increasing the agent population reduces inconsistency and improves accuracy, whereas extending discussion rounds induces a detrimental \textit{over-deliberation drift} where agents reinforce each other's mistakes. Ultimately, our findings outline the practical boundaries and safety margins of deploying collaborative multi-agent systems in high-stakes legal reasoning environments.

Related papers

• Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation [6.5331167606521205]
  Multi-agent debate (MAD) improves the reasoning capabilities of large language models by having multiple agents iteratively refine their responses through discussion.<n>MAD suffers from a critical vulnerability known as shared misconception: when a majority of agents converge on an incorrect answer, the debate process tends to amplify rather than correct the error.<n>Existing methods primarily address peer skew but leave the agents' inherently biased concept priors unaddressed.<n>We propose R$2$-MAD, a framework that equips agents with an experience memory accumulated from past debates.
  arXiv  Detail & Related papers  (2026-09-03T10:05:01Z)
• Investigating Multi-Agent Deliberation in Law [0.0]
  We introduce two novel multi-agent frameworks inspired by courtroom procedures and legal argumentation.<n>Our experiments on both legal and non-legal benchmarks reveal that multi-agent frameworks achieve comparable overall performance to baseline large language models.<n>Our work positions multi-agent systems as a promising direction for AI in the legal domain, while demonstrating the potential of law-inspired multi-agent approaches for deliberation.
  arXiv  Detail & Related papers  (2026-06-29T20:56:37Z)
• Debate to Align: Reliable Entity Alignment through Two-Stage Multi-Agent Debate [13.025981335381637]
  We propose AgentEA, a reliable EA framework based on multi-agent debate.<n>AgentEA improves embedding quality through entity representation preference optimization.<n>It then introduces a two-stage multi-role debate mechanism consisting of lightweight debate verification and deep debate alignment.
  arXiv  Detail & Related papers  (2026-04-15T06:54:27Z)
• DynaDebate: Breaking Homogeneity in Multi-Agent Debate with Dynamic Path Generation [47.62978918069135]
  We introduce Dynamic Multi-Agent Debate (DynaDebate), which enhances the effectiveness of multi-agent debate through three key mechanisms.<n>Extensive experiments demonstrate that DynaDebate achieves superior performance across various benchmarks, surpassing existing state-of-the-art MAD methods.
  arXiv  Detail & Related papers  (2026-01-09T12:01:33Z)
• Tool-MAD: A Multi-Agent Debate Framework for Fact Verification with Diverse Tool Augmentation and Adaptive Retrieval [10.62333858188658]
  Multi-Agent Debate (MAD) systems aim to improve answer accuracy by enabling multiple LLM agents to engage in dialogue.<n>Existing MAD frameworks primarily rely on internal knowledge or static documents, making them vulnerable to hallucinations.<n>We propose Tool-MAD, a multi-agent debate framework that enhances factual verification by assigning each agent a distinct external tool.
  arXiv  Detail & Related papers  (2026-01-08T09:07:41Z)
• Towards Scalable Oversight with Collaborative Multi-Agent Debate in Error Detection [81.52796950244705]
  Self-diagnosis is unreliable on complex tasks unless aided by reliable external feedback.<n>We introduce a new collaborative MAD protocol, termed ColMAD, that reframes MAD as a non-zero sum game.<n>We show that ColMAD significantly outperforms previous competitive MAD by 19%.
  arXiv  Detail & Related papers  (2025-10-23T19:46:00Z)
• MADIAVE: Multi-Agent Debate for Implicit Attribute Value Extraction [52.89860691282002]
  Implicit Attribute Value Extraction (AVE) is essential for accurately representing products in e-commerce.<n>Despite advances in multimodal large language models (MLLMs), implicit AVE remains challenging due to the complexity of multidimensional data.<n>We introduce textscmodelname, a multi-agent debate framework that employs multiple MLLM agents to iteratively refine inferences.
  arXiv  Detail & Related papers  (2025-10-07T06:27:42Z)
• Peacemaker or Troublemaker: How Sycophancy Shapes Multi-Agent Debate [30.66779902590191]
  Large language models (LLMs) often display sycophancy, a tendency toward excessive agreeability.<n>LLMs' inherent sycophancy can collapse debates into premature consensus.
  arXiv  Detail & Related papers  (2025-09-27T02:27:13Z)
• Enhancing Multi-Agent Debate System Performance via Confidence Expression [55.34012400580016]
  Multi-Agent Debate (MAD) systems simulate human debate and thereby improve task performance.<n>Some Large Language Models (LLMs) possess superior knowledge or reasoning capabilities for specific tasks, but struggle to clearly communicate this advantage during debates.<n>Inappropriate confidence expression can cause agents in MAD systems to either stubbornly maintain incorrect beliefs or converge prematurely on suboptimal answers.<n>We develop ConfMAD, a MAD framework that integrates confidence expression throughout the debate process.
  arXiv  Detail & Related papers  (2025-09-17T14:34:27Z)
• Revisiting Multi-Agent Debate as Test-Time Scaling: A Systematic Study of Conditional Effectiveness [50.29739337771454]
  Multi-agent debate (MAD) approaches offer improved reasoning, robustness, and diverse perspectives over monolithic models.<n>This paper conceptualizes MAD as a test-time computational scaling technique, distinguished by collaborative refinement and diverse exploration capabilities.<n>We conduct a comprehensive empirical investigation comparing MAD with strong self-agent test-time scaling baselines on mathematical reasoning and safety-related tasks.
  arXiv  Detail & Related papers  (2025-05-29T01:02:55Z)
• Is Multi-Agent Debate (MAD) the Silver Bullet? An Empirical Analysis of MAD in Code Summarization and Translation [10.038721196640864]
  Multi-Agent Debate (MAD) systems enable structured debates among Large Language Models (LLMs)<n> MAD promotes divergent thinking through role-specific agents, dynamic interactions, and structured decision-making.<n>This study investigates MAD's effectiveness on two Software Engineering (SE) tasks.
  arXiv  Detail & Related papers  (2025-03-15T07:30:37Z)
• Large Multimodal Agents: A Survey [78.81459893884737]
  Large language models (LLMs) have achieved superior performance in powering text-based AI agents. There is an emerging research trend focused on extending these LLM-powered AI agents into the multimodal domain. This review aims to provide valuable insights and guidelines for future research in this rapidly evolving field.
  arXiv  Detail & Related papers  (2024-02-23T06:04:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.