GroupDebate: Enhancing the Efficiency of Multi-Agent Debate Using Group Discussion

• URL: http://arxiv.org/abs/2409.14051v1
• Date: Sat, 21 Sep 2024 07:49:38 GMT
• Title: GroupDebate: Enhancing the Efficiency of Multi-Agent Debate Using Group Discussion
• Authors: Tongxuan Liu, Xingyu Wang, Weizhe Huang, Wenjiang Xu, Yuting Zeng, Lei Jiang, Hailong Yang, Jing Li,
• Abstract summary: This paper proposes a method to significantly reduce token cost in multi-agent debates. Our method significantly enhances the performance and efficiency of interactions in the multi-agent debate.
• Score: 8.948702488582583
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In recent years, Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse NLP tasks. Extensive research has explored how to enhance the logical reasoning abilities such as Chain-of-Thought, Chain-of-Thought with Self-Consistency, Tree-Of-Thoughts, and multi-agent debates. In the context of multi-agent debates, significant performance improvements can be achieved with an increasing number of agents and debate rounds. However, the escalation in the number of agents and debate rounds can drastically raise the tokens cost of debates, thereby limiting the scalability of the multi-agent debate technique. To better harness the advantages of multi-agent debates in logical reasoning tasks, this paper proposes a method to significantly reduce token cost in multi-agent debates. This approach involves dividing all agents into multiple debate groups, with agents engaging in debates within their respective groups and sharing interim debate results between groups. Comparative experiments across multiple datasets have demonstrated that this method can reduce the total tokens by up to 51.7% during debates and while potentially enhancing accuracy by as much as 25%. Our method significantly enhances the performance and efficiency of interactions in the multi-agent debate.

Related papers

• ACC-Debate: An Actor-Critic Approach to Multi-Agent Debate [20.040543142468344]
  We propose ACC-Debate, an Actor-Critic based learning framework to produce a two-agent team specialized in debate. We demonstrate that ACC-Debate outperforms SotA debate techniques on a wide array of benchmarks.
  arXiv  Detail & Related papers  (2024-10-30T19:09:02Z)
• Diversity of Thought Elicits Stronger Reasoning Capabilities in Multi-Agent Debate Frameworks [0.0]
  Chain-of-thought prompting, self-verification, and multi-agent debate are proposed to improve the reasoning and factual accuracy of large language models. We find that multi-agent debate helps at any model scale, and that diversity of thought elicits stronger reasoning in debating LLMs.
  arXiv  Detail & Related papers  (2024-10-10T21:59:01Z)
• Can LLMs Beat Humans in Debating? A Dynamic Multi-agent Framework for Competitive Debate [22.813887723656023]
  Agent for Debate (Agent4Debate) is a dynamic multi-agent framework based on Large Language Models (LLMs) The evaluation employs the Debatrix automatic scoring system and professional human reviewers based on the established Debatrix-Elo and Human-Elo ranking. Experimental results indicate that the state-of-the-art Agent4Debate exhibits capabilities comparable to those of humans.
  arXiv  Detail & Related papers  (2024-08-08T14:02:45Z)
• DebUnc: Mitigating Hallucinations in Large Language Model Agent Communication with Uncertainty Estimations [52.242449026151846]
  DebUnc is a multi-agent debate framework that uses uncertainty metrics to assess agent confidence levels. We adapted the attention mechanism to adjust token weights based on confidence levels. Our evaluations show that attention-based methods are particularly effective.
  arXiv  Detail & Related papers  (2024-07-08T22:15:01Z)
• Improving Multi-Agent Debate with Sparse Communication Topology [9.041025703879905]
  Multi-agent debate has proven effective in improving large language models quality for reasoning and factuality tasks. In this paper, we investigate the effect of communication connectivity in multi-agent systems. Our experiments on GPT and Mistral models reveal that multi-agent debates leveraging sparse communication topology can achieve comparable or superior performance.
  arXiv  Detail & Related papers  (2024-06-17T17:33:09Z)
• Debatrix: Multi-dimensional Debate Judge with Iterative Chronological Analysis Based on LLM [51.43102092480804]
  Debatrix is an automated debate judge based on Large Language Models (LLMs) To align with real-world debate scenarios, we introduced the PanelBench benchmark, comparing our system's performance to actual debate outcomes. The findings indicate a notable enhancement over directly using LLMs for debate evaluation.
  arXiv  Detail & Related papers  (2024-03-12T18:19:47Z)
• Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key? [84.36332588191623]
  We propose a novel group discussion framework to enrich the set of discussion mechanisms. We observe that the multi-agent discussion performs better than a single agent only when there is no demonstration in the prompt.
  arXiv  Detail & Related papers  (2024-02-28T12:04:05Z)
• Large Multimodal Agents: A Survey [78.81459893884737]
  Large language models (LLMs) have achieved superior performance in powering text-based AI agents. There is an emerging research trend focused on extending these LLM-powered AI agents into the multimodal domain. This review aims to provide valuable insights and guidelines for future research in this rapidly evolving field.
  arXiv  Detail & Related papers  (2024-02-23T06:04:23Z)
• Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs [7.7433783185451075]
  We benchmark a range of debating and prompting strategies to explore the trade-offs between cost, time, and accuracy. We find that multi-agent debating systems, in their current form, do not reliably outperform other proposed prompting strategies. We build on these results to offer insights into improving debating strategies, such as adjusting agent agreement levels.
  arXiv  Detail & Related papers  (2023-11-29T05:54:41Z)
• On the Discussion of Large Language Models: Symmetry of Agents and Interplay with Prompts [51.3324922038486]
  This paper reports the empirical results of the interplay of prompts and discussion mechanisms. It also proposes a scalable discussion mechanism based on conquer and merge.
  arXiv  Detail & Related papers  (2023-11-13T04:56:48Z)
• ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate [57.71597869337909]
  We build a multi-agent referee team called ChatEval to autonomously discuss and evaluate the quality of generated responses from different models. Our analysis shows that ChatEval transcends mere textual scoring, offering a human-mimicking evaluation process for reliable assessments.
  arXiv  Detail & Related papers  (2023-08-14T15:13:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.