Grow-and-Clip: Informative-yet-Concise Evidence Distillation for Answer Explanation

• URL: http://arxiv.org/abs/2201.05088v2
• Date: Fri, 14 Jan 2022 05:46:04 GMT
• Title: Grow-and-Clip: Informative-yet-Concise Evidence Distillation for Answer Explanation
• Authors: Yuyan Chen, Yanghua Xiao, Bang Liu
• Abstract summary: We argue that the evidences of an answer is critical to enhancing the interpretability of QA models. We are the first to explicitly define the concept of evidence as the supporting facts in a context which are informative, concise, and readable. We propose Grow-and-Clip Evidence Distillation (GCED) algorithm to extract evidences from the contexts by trade-off informativeness, conciseness, and readability.
• Score: 22.20733260041759
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Interpreting the predictions of existing Question Answering (QA) models is critical to many real-world intelligent applications, such as QA systems for healthcare, education, and finance. However, existing QA models lack interpretability and provide no feedback or explanation for end-users to help them understand why a specific prediction is the answer to a question. In this research, we argue that the evidences of an answer is critical to enhancing the interpretability of QA models. Unlike previous research that simply extracts several sentence(s) in the context as evidence, we are the first to explicitly define the concept of evidence as the supporting facts in a context which are informative, concise, and readable. Besides, we provide effective strategies to quantitatively measure the informativeness, conciseness and readability of evidence. Furthermore, we propose Grow-and-Clip Evidence Distillation (GCED) algorithm to extract evidences from the contexts by trade-off informativeness, conciseness, and readability. We conduct extensive experiments on the SQuAD and TriviaQA datasets with several baseline models to evaluate the effect of GCED on interpreting answers to questions. Human evaluation are also carried out to check the quality of distilled evidences. Experimental results show that automatic distilled evidences have human-like informativeness, conciseness and readability, which can enhance the interpretability of the answers to questions.

Related papers

• Retrieve to Explain: Evidence-driven Predictions with Language Models [0.791663505497707]
  We introduce Retrieve to Explain (R2E), a retrieval-based language model. R2E scores and ranks all possible answers to a research question based on evidence retrieved from a document corpus. We assess on the challenging task of drug target identification from scientific literature.
  arXiv  Detail & Related papers  (2024-02-06T15:13:17Z)
• InfoLossQA: Characterizing and Recovering Information Loss in Text Simplification [60.10193972862099]
  This work proposes a framework to characterize and recover simplification-induced information loss in form of question-and-answer pairs. QA pairs are designed to help readers deepen their knowledge of a text.
  arXiv  Detail & Related papers  (2024-01-29T19:00:01Z)
• CREPE: Open-Domain Question Answering with False Presuppositions [92.20501870319765]
  We introduce CREPE, a QA dataset containing a natural distribution of presupposition failures from online information-seeking forums. We find that 25% of questions contain false presuppositions, and provide annotations for these presuppositions and their corrections. We show that adaptations of existing open-domain QA models can find presuppositions moderately well, but struggle when predicting whether a presupposition is factually correct.
  arXiv  Detail & Related papers  (2022-11-30T18:54:49Z)
• ReasonChainQA: Text-based Complex Question Answering with Explainable Evidence Chains [15.837457557803507]
  We present a benchmark textbfReasonChainQA with explanatory and explicit evidence chains. ReasonChainQA consists of two subtasks: answer generation and evidence chains extraction, it also contains higher diversity for multi-hop questions. Additional experiment on supervised and unsupervised retrieval fully indicates the significance of ReasonChainQA.
  arXiv  Detail & Related papers  (2022-10-17T06:07:39Z)
• Do Answers to Boolean Questions Need Explanations? Yes [11.226970608525596]
  We release a new set of annotations marking the evidence in existing TyDi QA and BoolQ datasets. We show that our annotations can be used to train a model that extracts improved evidence spans.
  arXiv  Detail & Related papers  (2021-12-14T22:40:28Z)
• Introspective Distillation for Robust Question Answering [70.18644911309468]
  Question answering (QA) models are well-known to exploit data bias, e.g., the language prior in visual QA and the position bias in reading comprehension. Recent debiasing methods achieve good out-of-distribution (OOD) generalizability with a considerable sacrifice of the in-distribution (ID) performance. We present a novel debiasing method called Introspective Distillation (IntroD) to make the best of both worlds for QA.
  arXiv  Detail & Related papers  (2021-11-01T15:30:15Z)
• REM-Net: Recursive Erasure Memory Network for Commonsense Evidence Refinement [130.8875535449478]
  REM-Net is equipped with a module to refine the evidence by erasing the low-quality evidence that does not explain the question answering. Instead of retrieving evidence from existing knowledge bases, REM-Net leverages a pre-trained generative model to generate candidate evidence customized for the question. The results demonstrate the performance of REM-Net and show that the refined evidence is explainable.
  arXiv  Detail & Related papers  (2020-12-24T10:07:32Z)
• Context Modeling with Evidence Filter for Multiple Choice Question Answering [18.154792554957595]
  Multiple-Choice Question Answering (MCQA) is a challenging task in machine reading comprehension. The main challenge is to extract "evidence" from the given context that supports the correct answer. Existing work tackles this problem by annotated evidence or distant supervision with rules which overly rely on human efforts. We propose a simple yet effective approach termed evidence filtering to model the relationships between the encoded contexts.
  arXiv  Detail & Related papers  (2020-10-06T11:53:23Z)
• QED: A Framework and Dataset for Explanations in Question Answering [27.85923397716627]
  We release an expert-annotated dataset of QED explanations built upon a subset of the Google Natural Questions dataset. A promising result suggests that training on a relatively small amount of QED data can improve question answering.
  arXiv  Detail & Related papers  (2020-09-08T23:34:18Z)
• Interpretable Multi-Step Reasoning with Knowledge Extraction on Complex Healthcare Question Answering [89.76059961309453]
  HeadQA dataset contains multiple-choice questions authorized for the public healthcare specialization exam. These questions are the most challenging for current QA systems. We present a Multi-step reasoning with Knowledge extraction framework (MurKe) We are striving to make full use of off-the-shelf pre-trained models.
  arXiv  Detail & Related papers  (2020-08-06T02:47:46Z)
• Asking and Answering Questions to Evaluate the Factual Consistency of Summaries [80.65186293015135]
  We propose an automatic evaluation protocol called QAGS (pronounced "kags") to identify factual inconsistencies in a generated summary. QAGS is based on the intuition that if we ask questions about a summary and its source, we will receive similar answers if the summary is factually consistent with the source. We believe QAGS is a promising tool in automatically generating usable and factually consistent text.
  arXiv  Detail & Related papers  (2020-04-08T20:01:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.