Do Answers to Boolean Questions Need Explanations? Yes

• URL: http://arxiv.org/abs/2112.07772v1
• Date: Tue, 14 Dec 2021 22:40:28 GMT
• Title: Do Answers to Boolean Questions Need Explanations? Yes
• Authors: Sara Rosenthal, Mihaela Bornea, Avirup Sil, Radu Florian, Scott McCarley
• Abstract summary: We release a new set of annotations marking the evidence in existing TyDi QA and BoolQ datasets. We show that our annotations can be used to train a model that extracts improved evidence spans.
• Score: 11.226970608525596
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing datasets that contain boolean questions, such as BoolQ and TYDI QA , provide the user with a YES/NO response to the question. However, a one word response is not sufficient for an explainable system. We promote explainability by releasing a new set of annotations marking the evidence in existing TyDi QA and BoolQ datasets. We show that our annotations can be used to train a model that extracts improved evidence spans compared to models that rely on existing resources. We confirm our findings with a user study which shows that our extracted evidence spans enhance the user experience. We also provide further insight into the challenges of answering boolean questions, such as passages containing conflicting YES and NO answers, and varying degrees of relevance of the predicted evidence.

Related papers

• Evidence-Enhanced Triplet Generation Framework for Hallucination Alleviation in Generative Question Answering [41.990482015732574]
  We propose a novel evidence-enhanced triplet generation framework, EATQA, to predict all the combinations of (Question, Evidence, Answer) triplet. We bridge the distribution gap to distill the knowledge from evidence in inference stage. Our framework ensures the model to learn the logical relation between query, evidence and answer, which simultaneously improves the evidence generation and query answering.
  arXiv  Detail & Related papers  (2024-08-27T13:07:07Z)
• Open-Set Knowledge-Based Visual Question Answering with Inference Paths [79.55742631375063]
  The purpose of Knowledge-Based Visual Question Answering (KB-VQA) is to provide a correct answer to the question with the aid of external knowledge bases. We propose a new retriever-ranker paradigm of KB-VQA, Graph pATH rankER (GATHER for brevity) Specifically, it contains graph constructing, pruning, and path-level ranking, which not only retrieves accurate answers but also provides inference paths that explain the reasoning process.
  arXiv  Detail & Related papers  (2023-10-12T09:12:50Z)
• QACHECK: A Demonstration System for Question-Guided Multi-Hop Fact-Checking [68.06355980166053]
  We propose the Question-guided Multi-hop Fact-Checking (QACHECK) system. It guides the model's reasoning process by asking a series of questions critical for verifying a claim. It provides the source of evidence supporting each question, fostering a transparent, explainable, and user-friendly fact-checking process.
  arXiv  Detail & Related papers  (2023-10-11T15:51:53Z)
• CREPE: Open-Domain Question Answering with False Presuppositions [92.20501870319765]
  We introduce CREPE, a QA dataset containing a natural distribution of presupposition failures from online information-seeking forums. We find that 25% of questions contain false presuppositions, and provide annotations for these presuppositions and their corrections. We show that adaptations of existing open-domain QA models can find presuppositions moderately well, but struggle when predicting whether a presupposition is factually correct.
  arXiv  Detail & Related papers  (2022-11-30T18:54:49Z)
• ReasonChainQA: Text-based Complex Question Answering with Explainable Evidence Chains [15.837457557803507]
  We present a benchmark textbfReasonChainQA with explanatory and explicit evidence chains. ReasonChainQA consists of two subtasks: answer generation and evidence chains extraction, it also contains higher diversity for multi-hop questions. Additional experiment on supervised and unsupervised retrieval fully indicates the significance of ReasonChainQA.
  arXiv  Detail & Related papers  (2022-10-17T06:07:39Z)
• Grow-and-Clip: Informative-yet-Concise Evidence Distillation for Answer Explanation [22.20733260041759]
  We argue that the evidences of an answer is critical to enhancing the interpretability of QA models. We are the first to explicitly define the concept of evidence as the supporting facts in a context which are informative, concise, and readable. We propose Grow-and-Clip Evidence Distillation (GCED) algorithm to extract evidences from the contexts by trade-off informativeness, conciseness, and readability.
  arXiv  Detail & Related papers  (2022-01-13T17:18:17Z)
• REM-Net: Recursive Erasure Memory Network for Commonsense Evidence Refinement [130.8875535449478]
  REM-Net is equipped with a module to refine the evidence by erasing the low-quality evidence that does not explain the question answering. Instead of retrieving evidence from existing knowledge bases, REM-Net leverages a pre-trained generative model to generate candidate evidence customized for the question. The results demonstrate the performance of REM-Net and show that the refined evidence is explainable.
  arXiv  Detail & Related papers  (2020-12-24T10:07:32Z)
• Challenges in Information-Seeking QA: Unanswerable Questions and Paragraph Retrieval [46.3246135936476]
  We analyze why answering information-seeking queries is more challenging and where their prevalent unanswerabilities arise. Our controlled experiments suggest two headrooms -- paragraph selection and answerability prediction. We manually annotate 800 unanswerable examples across six languages on what makes them challenging to answer.
  arXiv  Detail & Related papers  (2020-10-22T17:48:17Z)
• SQuINTing at VQA Models: Introspecting VQA Models with Sub-Questions [66.86887670416193]
  We show that state-of-the-art VQA models have comparable performance in answering perception and reasoning questions, but suffer from consistency problems. To address this shortcoming, we propose an approach called Sub-Question-aware Network Tuning (SQuINT) We show that SQuINT improves model consistency by 5%, also marginally improving performance on the Reasoning questions in VQA, while also displaying better attention maps.
  arXiv  Detail & Related papers  (2020-01-20T01:02:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.