Do Multi-Hop Question Answering Systems Know How to Answer the Single-Hop Sub-Questions?

• URL: http://arxiv.org/abs/2002.09919v2
• Date: Wed, 27 Jan 2021 04:18:57 GMT
• Title: Do Multi-Hop Question Answering Systems Know How to Answer the Single-Hop Sub-Questions?
• Authors: Yixuan Tang, Hwee Tou Ng, Anthony K.H. Tung
• Abstract summary: We investigate whether top-performing models for multi-hop questions understand the underlying sub-questions like humans. We show that multiple state-of-the-art multi-hop QA models fail to correctly answer a large portion of sub-questions. Our work takes a step forward towards building a more explainable multi-hop QA system.
• Score: 23.991872322492384
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-hop question answering (QA) requires a model to retrieve and integrate information from different parts of a long text to answer a question. Humans answer this kind of complex questions via a divide-and-conquer approach. In this paper, we investigate whether top-performing models for multi-hop questions understand the underlying sub-questions like humans. We adopt a neural decomposition model to generate sub-questions for a multi-hop complex question, followed by extracting the corresponding sub-answers. We show that multiple state-of-the-art multi-hop QA models fail to correctly answer a large portion of sub-questions, although their corresponding multi-hop questions are correctly answered. This indicates that these models manage to answer the multi-hop questions using some partial clues, instead of truly understanding the reasoning paths. We also propose a new model which significantly improves the performance on answering the sub-questions. Our work takes a step forward towards building a more explainable multi-hop QA system.

Related papers

• Understanding and Improving Zero-shot Multi-hop Reasoning in Generative Question Answering [85.79940770146557]
  We decompose multi-hop questions into multiple corresponding single-hop questions. We find marked inconsistency in QA models' answers on these pairs of ostensibly identical question chains. When trained only on single-hop questions, models generalize poorly to multi-hop questions.
  arXiv  Detail & Related papers  (2022-10-09T11:48:07Z)
• Prompt-based Conservation Learning for Multi-hop Question Answering [11.516763652013005]
  Multi-hop question answering requires reasoning over multiple documents to answer a complex question. Most existing multi-hop QA methods fail to answer a large fraction of sub-questions. We propose the Prompt-based Conservation Learning framework for multi-hop QA.
  arXiv  Detail & Related papers  (2022-09-14T20:50:46Z)
• Locate Then Ask: Interpretable Stepwise Reasoning for Multi-hop Question Answering [71.49131159045811]
  Multi-hop reasoning requires aggregating multiple documents to answer a complex question. Existing methods usually decompose the multi-hop question into simpler single-hop questions. We propose an interpretable stepwise reasoning framework to incorporate both single-hop supporting sentence identification and single-hop question generation.
  arXiv  Detail & Related papers  (2022-08-22T13:24:25Z)
• Interpretable AMR-Based Question Decomposition for Multi-hop Question Answering [12.35571328854374]
  We propose a Question Decomposition method based on Abstract Meaning Representation (QDAMR) for multi-hop QA. We decompose a multi-hop question into simpler sub-questions and answer them in order. Experimental results on HotpotQA demonstrate that our approach is competitive for interpretable reasoning.
  arXiv  Detail & Related papers  (2022-06-16T23:46:33Z)
• Modeling Multi-hop Question Answering as Single Sequence Prediction [88.72621430714985]
  We propose a simple generative approach (PathFid) that extends the task beyond just answer generation. PathFid explicitly models the reasoning process to resolve the answer for multi-hop questions. Our experiments demonstrate that PathFid leads to strong performance gains on two multi-hop QA datasets.
  arXiv  Detail & Related papers  (2022-05-18T21:57:59Z)
• Calibrating Trust of Multi-Hop Question Answering Systems with Decompositional Probes [14.302797773412543]
  Multi-hop Question Answering (QA) is a challenging task since it requires an accurate aggregation of information from multiple context paragraphs. Recent work in multi-hop QA has shown that performance can be boosted by first decomposing the questions into simpler, single-hop questions. We show that decomposition is an effective form of probing QA systems as well as a promising approach to explanation generation.
  arXiv  Detail & Related papers  (2022-04-16T01:03:36Z)
• Unsupervised Multi-hop Question Answering by Question Generation [108.61653629883753]
  MQA-QG is an unsupervised framework that can generate human-like multi-hop training data. Using only generated training data, we can train a competent multi-hop QA which achieves 61% and 83% of the supervised learning performance.
  arXiv  Detail & Related papers  (2020-10-23T19:13:47Z)
• Reinforced Multi-task Approach for Multi-hop Question Generation [47.15108724294234]
  We take up Multi-hop question generation, which aims at generating relevant questions based on supporting facts in the context. We employ multitask learning with the auxiliary task of answer-aware supporting fact prediction to guide the question generator. We demonstrate the effectiveness of our approach through experiments on the multi-hop question answering dataset, HotPotQA.
  arXiv  Detail & Related papers  (2020-04-05T10:16:59Z)
• Unsupervised Question Decomposition for Question Answering [102.56966847404287]
  We propose an algorithm for One-to-N Unsupervised Sequence Sequence (ONUS) that learns to map one hard, multi-hop question to many simpler, single-hop sub-questions. We show large QA improvements on HotpotQA over a strong baseline on the original, out-of-domain, and multi-hop dev sets.
  arXiv  Detail & Related papers  (2020-02-22T19:40:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.