Context Modeling with Evidence Filter for Multiple Choice Question Answering

• URL: http://arxiv.org/abs/2010.02649v1
• Date: Tue, 6 Oct 2020 11:53:23 GMT
• Title: Context Modeling with Evidence Filter for Multiple Choice Question Answering
• Authors: Sicheng Yu, Hao Zhang, Wei Jing, Jing Jiang
• Abstract summary: Multiple-Choice Question Answering (MCQA) is a challenging task in machine reading comprehension. The main challenge is to extract "evidence" from the given context that supports the correct answer. Existing work tackles this problem by annotated evidence or distant supervision with rules which overly rely on human efforts. We propose a simple yet effective approach termed evidence filtering to model the relationships between the encoded contexts.
• Score: 18.154792554957595
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multiple-Choice Question Answering (MCQA) is a challenging task in machine reading comprehension. The main challenge in MCQA is to extract "evidence" from the given context that supports the correct answer. In the OpenbookQA dataset, the requirement of extracting "evidence" is particularly important due to the mutual independence of sentences in the context. Existing work tackles this problem by annotated evidence or distant supervision with rules which overly rely on human efforts. To address the challenge, we propose a simple yet effective approach termed evidence filtering to model the relationships between the encoded contexts with respect to different options collectively and to potentially highlight the evidence sentences and filter out unrelated sentences. In addition to the effective reduction of human efforts of our approach compared, through extensive experiments on OpenbookQA, we show that the proposed approach outperforms the models that use the same backbone and more training data; and our parameter analysis also demonstrates the interpretability of our approach.

Related papers

• Optimizing Language Model's Reasoning Abilities with Weak Supervision [48.60598455782159]
  We present textscPuzzleBen, a weakly supervised benchmark that comprises 25,147 complex questions, answers, and human-generated rationales. A unique aspect of our dataset is the inclusion of 10,000 unannotated questions, enabling us to explore utilizing fewer supersized data to boost LLMs' inference capabilities.
  arXiv  Detail & Related papers  (2024-05-07T07:39:15Z)
• Unsupervised Candidate Answer Extraction through Differentiable Masker-Reconstructor Model [21.667471025804936]
  We propose a novel unsupervised candidate answer extraction approach that leverages the inherent structure of context passages through a Differentiable Masker-Reconstructor (DMR) Model. We benchmark a comprehensive set of supervised and unsupervised candidate answer extraction methods. We demonstrate the effectiveness of the DMR model by showing its performance is superior among unsupervised methods and comparable to supervised methods.
  arXiv  Detail & Related papers  (2023-10-19T19:07:08Z)
• Progressive Evidence Refinement for Open-domain Multimodal Retrieval Question Answering [20.59485758381809]
  Current multimodal retrieval question-answering models face two main challenges. utilizing compressed evidence features as input to the model results in the loss of fine-grained information within the evidence. We propose a two-stage framework for evidence retrieval and question-answering to alleviate these issues.
  arXiv  Detail & Related papers  (2023-10-15T01:18:39Z)
• Momentum Contrastive Pre-training for Question Answering [54.57078061878619]
  MCROSS introduces a momentum contrastive learning framework to align the answer probability between cloze-like and natural query-passage sample pairs. Our method achieves noticeable improvement compared with all baselines in both supervised and zero-shot scenarios.
  arXiv  Detail & Related papers  (2022-12-12T08:28:22Z)
• elBERto: Self-supervised Commonsense Learning for Question Answering [131.51059870970616]
  We propose a Self-supervised Bidirectional Representation Learning of Commonsense framework, which is compatible with off-the-shelf QA model architectures. The framework comprises five self-supervised tasks to force the model to fully exploit the additional training signals from contexts containing rich commonsense. elBERto achieves substantial improvements on out-of-paragraph and no-effect questions where simple lexical similarity comparison does not help.
  arXiv  Detail & Related papers  (2022-03-17T16:23:45Z)
• Grow-and-Clip: Informative-yet-Concise Evidence Distillation for Answer Explanation [22.20733260041759]
  We argue that the evidences of an answer is critical to enhancing the interpretability of QA models. We are the first to explicitly define the concept of evidence as the supporting facts in a context which are informative, concise, and readable. We propose Grow-and-Clip Evidence Distillation (GCED) algorithm to extract evidences from the contexts by trade-off informativeness, conciseness, and readability.
  arXiv  Detail & Related papers  (2022-01-13T17:18:17Z)
• SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction [51.27558374091491]
  We propose to explicitly teach the model to capture relevant contexts and entity types by supervising and augmenting intermediate steps (SAIS) for relation extraction. Based on a broad spectrum of carefully designed tasks, our proposed SAIS method not only extracts relations of better quality due to more effective supervision, but also retrieves the corresponding supporting evidence more accurately.
  arXiv  Detail & Related papers  (2021-09-24T17:37:35Z)
• EviDR: Evidence-Emphasized Discrete Reasoning for Reasoning Machine Reading Comprehension [39.970232108247394]
  Reasoning machine reading comprehension (R-MRC) aims to answer complex questions that require discrete reasoning based on text. Previous end-to-end methods that achieve state-of-the-art performance rarely solve the problem by paying enough emphasis on the modeling of evidence. We propose an evidence-emphasized discrete reasoning approach (EviDR), in which sentence and clause level evidence is first detected based on distant supervision.
  arXiv  Detail & Related papers  (2021-08-18T06:49:58Z)
• Discrete Reasoning Templates for Natural Language Understanding [79.07883990966077]
  We present an approach that reasons about complex questions by decomposing them to simpler subquestions. We derive the final answer according to instructions in a predefined reasoning template. We show that our approach is competitive with the state-of-the-art while being interpretable and requires little supervision.
  arXiv  Detail & Related papers  (2021-04-05T18:56:56Z)
• A Wrong Answer or a Wrong Question? An Intricate Relationship between Question Reformulation and Answer Selection in Conversational Question Answering [15.355557454305776]
  We show that question rewriting (QR) of the conversational context allows to shed more light on this phenomenon. We present the results of this analysis on the TREC CAsT and QuAC (CANARD) datasets.
  arXiv  Detail & Related papers  (2020-10-13T06:29:51Z)
• Knowledgeable Dialogue Reading Comprehension on Key Turns [84.1784903043884]
  Multi-choice machine reading comprehension (MRC) requires models to choose the correct answer from candidate options given a passage and a question. Our research focuses dialogue-based MRC, where the passages are multi-turn dialogues. It suffers from two challenges, the answer selection decision is made without support of latently helpful commonsense, and the multi-turn context may hide considerable irrelevant information.
  arXiv  Detail & Related papers  (2020-04-29T07:04:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.