Unification-based Reconstruction of Multi-hop Explanations for Science Questions

• URL: http://arxiv.org/abs/2004.00061v2
• Date: Wed, 10 Feb 2021 09:32:05 GMT
• Title: Unification-based Reconstruction of Multi-hop Explanations for Science Questions
• Authors: Marco Valentino, Mokanarangan Thayaparan, Andr\'e Freitas
• Abstract summary: We propose a method to leverage explanatory patterns emerging in a corpus of scientific explanations. The framework ranks a set of atomic facts by integrating lexical relevance with the notion of unification power. An extensive evaluation is performed on the Worldtree corpus, integrating k-NN clustering and Information Retrieval (IR) techniques.
• Score: 4.726777092009554
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents a novel framework for reconstructing multi-hop explanations in science Question Answering (QA). While existing approaches for multi-hop reasoning build explanations considering each question in isolation, we propose a method to leverage explanatory patterns emerging in a corpus of scientific explanations. Specifically, the framework ranks a set of atomic facts by integrating lexical relevance with the notion of unification power, estimated analysing explanations for similar questions in the corpus. An extensive evaluation is performed on the Worldtree corpus, integrating k-NN clustering and Information Retrieval (IR) techniques. We present the following conclusions: (1) The proposed method achieves results competitive with Transformers, yet being orders of magnitude faster, a feature that makes it scalable to large explanatory corpora (2) The unification-based mechanism has a key role in reducing semantic drift, contributing to the reconstruction of many hops explanations (6 or more facts) and the ranking of complex inference facts (+12.0 Mean Average Precision) (3) Crucially, the constructed explanations can support downstream QA models, improving the accuracy of BERT by up to 10% overall.

Related papers

• Make LLMs better zero-shot reasoners: Structure-orientated autonomous reasoning [52.83539473110143]
  We introduce a novel structure-oriented analysis method to help Large Language Models (LLMs) better understand a question. To further improve the reliability in complex question-answering tasks, we propose a multi-agent reasoning system, Structure-oriented Autonomous Reasoning Agents (SARA) Extensive experiments verify the effectiveness of the proposed reasoning system. Surprisingly, in some cases, the system even surpasses few-shot methods.
  arXiv  Detail & Related papers  (2024-10-18T05:30:33Z)
• Towards a Framework for Evaluating Explanations in Automated Fact Verification [12.904145308839997]
  As deep neural models in NLP become more complex, the necessity to interpret them becomes greater. A burgeoning interest has emerged in rationalizing explanations to provide short and coherent justifications for predictions. We advocate for a formal framework for key concepts and properties about rationalizing explanations to support their evaluation systematically.
  arXiv  Detail & Related papers  (2024-03-29T17:50:28Z)
• Leveraging Structured Information for Explainable Multi-hop Question Answering and Reasoning [14.219239732584368]
  In this work, we investigate constructing and leveraging extracted semantic structures (graphs) for multi-hop question answering. Empirical results and human evaluations show that our framework: generates more faithful reasoning chains and substantially improves the QA performance on two benchmark datasets.
  arXiv  Detail & Related papers  (2023-11-07T05:32:39Z)
• Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering [124.16250115608604]
  We present Science Question Answering (SQA), a new benchmark that consists of 21k multimodal multiple choice questions with a diverse set of science topics and annotations of their answers with corresponding lectures and explanations. We show that SQA improves the question answering performance by 1.20% in few-shot GPT-3 and 3.99% in fine-tuned UnifiedQA. Our analysis further shows that language models, similar to humans, benefit from explanations to learn from fewer data and achieve the same performance with just 40% of the data.
  arXiv  Detail & Related papers  (2022-09-20T07:04:24Z)
• Entailment Tree Explanations via Iterative Retrieval-Generation Reasoner [56.08919422452905]
  We propose an architecture called Iterative Retrieval-Generation Reasoner (IRGR) Our model is able to explain a given hypothesis by systematically generating a step-by-step explanation from textual premises. We outperform existing benchmarks on premise retrieval and entailment tree generation, with around 300% gain in overall correctness.
  arXiv  Detail & Related papers  (2022-05-18T21:52:11Z)
• Detection Accuracy for Evaluating Compositional Explanations of Units [5.220940151628734]
  Two examples of methods that use this approach are Network Dissection and Compositional explanations. While intuitively, logical forms are more informative than atomic concepts, it is not clear how to quantify this improvement. We propose to use as evaluation metric the Detection Accuracy, which measures units' consistency of detection of their assigned explanations.
  arXiv  Detail & Related papers  (2021-09-16T08:47:34Z)
• Local Explanation of Dialogue Response Generation [77.68077106724522]
  Local explanation of response generation (LERG) is proposed to gain insights into the reasoning process of a generation model. LERG views the sequence prediction as uncertainty estimation of a human response and then creates explanations by perturbing the input and calculating the certainty change over the human response. Our results show that our method consistently improves other widely used methods on proposed automatic- and human- evaluation metrics for this new task by 4.4-12.8%.
  arXiv  Detail & Related papers  (2021-06-11T17:58:36Z)
• Dynamic Semantic Graph Construction and Reasoning for Explainable Multi-hop Science Question Answering [50.546622625151926]
  We propose a new framework to exploit more valid facts while obtaining explainability for multi-hop QA. Our framework contains three new ideas: (a) tt AMR-SG, an AMR-based Semantic Graph, constructed by candidate fact AMRs to uncover any hop relations among question, answer and multiple facts, (b) a novel path-based fact analytics approach exploiting tt AMR-SG to extract active facts from a large fact pool to answer questions, and (c) a fact-level relation modeling leveraging graph convolution network (GCN) to guide the reasoning process.
  arXiv  Detail & Related papers  (2021-05-25T09:14:55Z)
• Discrete Reasoning Templates for Natural Language Understanding [79.07883990966077]
  We present an approach that reasons about complex questions by decomposing them to simpler subquestions. We derive the final answer according to instructions in a predefined reasoning template. We show that our approach is competitive with the state-of-the-art while being interpretable and requires little supervision.
  arXiv  Detail & Related papers  (2021-04-05T18:56:56Z)
• ExplanationLP: Abductive Reasoning for Explainable Science Question Answering [4.726777092009554]
  This paper frames question answering as an abductive reasoning problem. We construct plausible explanations for each choice and then selecting the candidate with the best explanation as the final answer. Our system, ExplanationLP, elicits explanations by constructing a weighted graph of relevant facts for each candidate answer.
  arXiv  Detail & Related papers  (2020-10-25T14:49:24Z)
• Multi-hop Inference for Question-driven Summarization [39.08269647808958]
  We propose a novel question-driven abstractive summarization method, Multi-hop Selective Generator (MSG) MSG incorporates multi-hop reasoning into question-driven summarization and, meanwhile, provide justifications for the generated summaries. Experimental results show that the proposed method consistently outperforms state-of-the-art methods on two non-factoid QA datasets.
  arXiv  Detail & Related papers  (2020-10-08T02:36:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.