BELLE: A Bi-Level Multi-Agent Reasoning Framework for Multi-Hop Question Answering

• URL: http://arxiv.org/abs/2505.11811v1
• Date: Sat, 17 May 2025 03:43:30 GMT
• Title: BELLE: A Bi-Level Multi-Agent Reasoning Framework for Multi-Hop Question Answering
• Authors: Taolin Zhang, Dongyang Li, Qizhou Chen, Chengyu Wang, Xiaofeng He,
• Abstract summary: Multi-hop question answering (QA) involves finding multiple relevant passages and performing step-by-step reasoning to answer complex questions.<n>Previous works on multi-hop QA employ specific methods from different modeling perspectives based on large language models (LLMs)<n>We propose a Bi-levEL muLti-agEnt reasoning (BELLE) framework to address multi-hop QA by specifically focusing on the correspondence between question types and methods.
• Score: 23.40379713863448
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-hop question answering (QA) involves finding multiple relevant passages and performing step-by-step reasoning to answer complex questions. Previous works on multi-hop QA employ specific methods from different modeling perspectives based on large language models (LLMs), regardless of the question types. In this paper, we first conduct an in-depth analysis of public multi-hop QA benchmarks, dividing the questions into four types and evaluating five types of cutting-edge methods for multi-hop QA: Chain-of-Thought (CoT), Single-step, Iterative-step, Sub-step, and Adaptive-step. We find that different types of multi-hop questions have varying degrees of sensitivity to different types of methods. Thus, we propose a Bi-levEL muLti-agEnt reasoning (BELLE) framework to address multi-hop QA by specifically focusing on the correspondence between question types and methods, where each type of method is regarded as an ''operator'' by prompting LLMs differently. The first level of BELLE includes multiple agents that debate to obtain an executive plan of combined ''operators'' to address the multi-hop QA task comprehensively. During the debate, in addition to the basic roles of affirmative debater, negative debater, and judge, at the second level, we further leverage fast and slow debaters to monitor whether changes in viewpoints are reasonable. Extensive experiments demonstrate that BELLE significantly outperforms strong baselines in various datasets. Additionally, the model consumption of BELLE is higher cost-effectiveness than that of single models in more complex multi-hop QA scenarios.

Related papers

• GenDec: A robust generative Question-decomposition method for Multi-hop reasoning [32.12904215053187]
  Multi-hop QA involves step-by-step reasoning to answer complex questions. Existing large language models'(LLMs) reasoning ability in multi-hop question answering remains exploration. It is unclear whether LLMs follow a desired reasoning chain to reach the right final answer.
  arXiv  Detail & Related papers  (2024-02-17T02:21:44Z)
• End-to-End Beam Retrieval for Multi-Hop Question Answering [37.13580394608824]
  Multi-hop question answering involves finding multiple relevant passages and step-by-step reasoning to answer complex questions. Previous retrievers were customized for two-hop questions, and most of them were trained separately across different hops. We introduce Beam Retrieval, an end-to-end beam retrieval framework for multi-hop QA.
  arXiv  Detail & Related papers  (2023-08-17T13:24:14Z)
• Performance Prediction for Multi-hop Questions [7.388002745070808]
  We propose multHP, a novel pre-retrieval method for predicting the performance of open-domain multi-hop questions. Our evaluation shows that the proposed model is a strong predictor of the performance, outperforming traditional single-hop QPP models.
  arXiv  Detail & Related papers  (2023-08-12T01:34:41Z)
• Understanding and Improving Zero-shot Multi-hop Reasoning in Generative Question Answering [85.79940770146557]
  We decompose multi-hop questions into multiple corresponding single-hop questions. We find marked inconsistency in QA models' answers on these pairs of ostensibly identical question chains. When trained only on single-hop questions, models generalize poorly to multi-hop questions.
  arXiv  Detail & Related papers  (2022-10-09T11:48:07Z)
• Prompt-based Conservation Learning for Multi-hop Question Answering [11.516763652013005]
  Multi-hop question answering requires reasoning over multiple documents to answer a complex question. Most existing multi-hop QA methods fail to answer a large fraction of sub-questions. We propose the Prompt-based Conservation Learning framework for multi-hop QA.
  arXiv  Detail & Related papers  (2022-09-14T20:50:46Z)
• Locate Then Ask: Interpretable Stepwise Reasoning for Multi-hop Question Answering [71.49131159045811]
  Multi-hop reasoning requires aggregating multiple documents to answer a complex question. Existing methods usually decompose the multi-hop question into simpler single-hop questions. We propose an interpretable stepwise reasoning framework to incorporate both single-hop supporting sentence identification and single-hop question generation.
  arXiv  Detail & Related papers  (2022-08-22T13:24:25Z)
• Modeling Multi-hop Question Answering as Single Sequence Prediction [88.72621430714985]
  We propose a simple generative approach (PathFid) that extends the task beyond just answer generation. PathFid explicitly models the reasoning process to resolve the answer for multi-hop questions. Our experiments demonstrate that PathFid leads to strong performance gains on two multi-hop QA datasets.
  arXiv  Detail & Related papers  (2022-05-18T21:57:59Z)
• Ask to Understand: Question Generation for Multi-hop Question Answering [11.626390908264872]
  Multi-hop Question Answering (QA) requires the machine to answer complex questions by finding scattering clues and reasoning from multiple documents. We propose a novel method to complete multi-hop QA from the perspective of Question Generation (QG)
  arXiv  Detail & Related papers  (2022-03-17T04:02:29Z)
• QA4QG: Using Question Answering to Constrain Multi-Hop Question Generation [54.136509061542775]
  Multi-hop question generation (MQG) aims to generate complex questions which require reasoning over multiple pieces of information of the input passage. We propose a novel framework, QA4QG, a QA-augmented BART-based framework for MQG. Our results on the HotpotQA dataset show that QA4QG outperforms all state-of-the-art models.
  arXiv  Detail & Related papers  (2022-02-14T08:16:47Z)
• Generative Context Pair Selection for Multi-hop Question Answering [60.74354009152721]
  We propose a generative context selection model for multi-hop question answering. Our proposed generative passage selection model has a better performance (4.9% higher than baseline) on adversarial held-out set.
  arXiv  Detail & Related papers  (2021-04-18T07:00:48Z)
• Reinforced Multi-task Approach for Multi-hop Question Generation [47.15108724294234]
  We take up Multi-hop question generation, which aims at generating relevant questions based on supporting facts in the context. We employ multitask learning with the auxiliary task of answer-aware supporting fact prediction to guide the question generator. We demonstrate the effectiveness of our approach through experiments on the multi-hop question answering dataset, HotPotQA.
  arXiv  Detail & Related papers  (2020-04-05T10:16:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.