Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?

• URL: http://arxiv.org/abs/2402.18272v1
• Date: Wed, 28 Feb 2024 12:04:05 GMT
• Title: Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?
• Authors: Qineng Wang, Zihao Wang, Ying Su, Hanghang Tong, Yangqiu Song
• Abstract summary: We propose a novel group discussion framework to enrich the set of discussion mechanisms. We observe that the multi-agent discussion performs better than a single agent only when there is no demonstration in the prompt.
• Score: 84.36332588191623
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent progress in LLMs discussion suggests that multi-agent discussion improves the reasoning abilities of LLMs. In this work, we reevaluate this claim through systematic experiments, where we propose a novel group discussion framework to enrich the set of discussion mechanisms. Interestingly, our results show that a single-agent LLM with strong prompts can achieve almost the same performance as the best existing discussion approach on a wide range of reasoning tasks and backbone LLMs. We observe that the multi-agent discussion performs better than a single agent only when there is no demonstration in the prompt. Further study reveals the common interaction mechanisms of LLMs during the discussion.

Related papers

• Intermittent Semi-working Mask: A New Masking Paradigm for LLMs [13.271151693864114]
  Multi-turn dialogues are a key interaction method between humans and Large Language Models (LLMs) We propose a novel masking scheme called Intermittent Semi-working Mask (ISM) to address these problems.
  arXiv  Detail & Related papers  (2024-08-01T13:22:01Z)
• Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration [70.09561665520043]
  We propose a novel framework for multi-agent collaboration that introduces Reinforced Advantage feedback (ReAd) for efficient self-refinement of plans. We provide theoretical analysis by extending advantage-weighted regression in reinforcement learning to multi-agent systems. Experiments on Over-AI and a difficult variant of RoCoBench show that ReAd surpasses baselines in success rate, and also significantly decreases the interaction steps of agents.
  arXiv  Detail & Related papers  (2024-05-23T08:33:19Z)
• LLM-based Multi-Agent Reinforcement Learning: Current and Future Directions [8.55917897789612]
  We focus on the cooperative tasks of multiple agents with a common goal and communication among them. We also consider human-in/on-the-loop scenarios enabled by the language component in the framework.
  arXiv  Detail & Related papers  (2024-05-17T22:10:23Z)
• LLM Discussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-Play [43.55248812883912]
  Large language models (LLMs) have shown exceptional proficiency in natural language processing but often fall short of generating creative and original responses to open-ended questions. We propose LLM Discussion, a three-phase discussion framework that facilitates vigorous and diverging idea exchanges. We evaluate the efficacy of the proposed framework with the Alternative Uses Test, Similarities Test, Instances Test, and Scientific Creativity Test.
  arXiv  Detail & Related papers  (2024-05-10T10:19:14Z)
• CoMM: Collaborative Multi-Agent, Multi-Reasoning-Path Prompting for Complex Problem Solving [9.446546965008249]
  We propose a collaborative multi-agent, multi-reasoning-path (CoMM) prompting framework. Specifically, we prompt LLMs to play different roles in a problem-solving team, and encourage different role-play agents to collaboratively solve the target task. Empirical results demonstrate the effectiveness of the proposed methods on two college-level science problems.
  arXiv  Detail & Related papers  (2024-04-26T23:29:12Z)
• Large Multimodal Agents: A Survey [78.81459893884737]
  Large language models (LLMs) have achieved superior performance in powering text-based AI agents. There is an emerging research trend focused on extending these LLM-powered AI agents into the multimodal domain. This review aims to provide valuable insights and guidelines for future research in this rapidly evolving field.
  arXiv  Detail & Related papers  (2024-02-23T06:04:23Z)
• Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves [57.974103113675795]
  We present a method named Rephrase and Respond' (RaR) which allows Large Language Models to rephrase and expand questions posed by humans. RaR serves as a simple yet effective prompting method for improving performance. We show that RaR is complementary to the popular Chain-of-Thought (CoT) methods, both theoretically and empirically.
  arXiv  Detail & Related papers  (2023-11-07T18:43:34Z)
• Position: Key Claims in LLM Research Have a Long Tail of Footnotes [81.14898541318198]
  We argue that we do not have a working definition of Large Language Models (LLMs) We critically examine five common claims regarding their properties. We conclude with suggestions for future research directions and their framing.
  arXiv  Detail & Related papers  (2023-08-14T13:00:53Z)
• Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate [85.3444184685235]
  We propose a Multi-Agent Debate (MAD) framework, in which multiple agents express their arguments in the state of "tit for tat" and a judge manages the debate process to obtain a final solution. Our framework encourages divergent thinking in LLMs which would be helpful for tasks that require deep levels of contemplation.
  arXiv  Detail & Related papers  (2023-05-30T15:25:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.