Retrospective Reader for Machine Reading Comprehension

• URL: http://arxiv.org/abs/2001.09694v4
• Date: Fri, 11 Dec 2020 09:28:12 GMT
• Title: Retrospective Reader for Machine Reading Comprehension
• Authors: Zhuosheng Zhang, Junjie Yang, Hai Zhao
• Abstract summary: Machine reading comprehension (MRC) is an AI challenge that requires machine to determine the correct answers to questions based on a given passage. When unanswerable questions are involved in the MRC task, an essential verification module called verifier is especially required in addition to the encoder. This paper devotes itself to exploring better verifier design for the MRC task with unanswerable questions.
• Score: 90.6069071495214
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine reading comprehension (MRC) is an AI challenge that requires machine to determine the correct answers to questions based on a given passage. MRC systems must not only answer question when necessary but also distinguish when no answer is available according to the given passage and then tactfully abstain from answering. When unanswerable questions are involved in the MRC task, an essential verification module called verifier is especially required in addition to the encoder, though the latest practice on MRC modeling still most benefits from adopting well pre-trained language models as the encoder block by only focusing on the "reading". This paper devotes itself to exploring better verifier design for the MRC task with unanswerable questions. Inspired by how humans solve reading comprehension questions, we proposed a retrospective reader (Retro-Reader) that integrates two stages of reading and verification strategies: 1) sketchy reading that briefly investigates the overall interactions of passage and question, and yield an initial judgment; 2) intensive reading that verifies the answer and gives the final prediction. The proposed reader is evaluated on two benchmark MRC challenge datasets SQuAD2.0 and NewsQA, achieving new state-of-the-art results. Significance tests show that our model is significantly better than the strong ELECTRA and ALBERT baselines. A series of analysis is also conducted to interpret the effectiveness of the proposed reader.

Related papers

• Towards Reliable and Factual Response Generation: Detecting Unanswerable Questions in Information-Seeking Conversations [16.99952884041096]
  Generative AI models face the challenge of hallucinations that can undermine users' trust in such systems. We approach the problem of conversational information seeking as a two-step process, where relevant passages in a corpus are identified first and then summarized into a final system response. Specifically, our proposed method employs a sentence-level classifier to detect if the answer is present, then aggregates these predictions on the passage level, and eventually across the top-ranked passages to arrive at a final answerability estimate.
  arXiv  Detail & Related papers  (2024-01-21T10:15:36Z)
• Re-Reading Improves Reasoning in Large Language Models [87.46256176508376]
  We introduce a simple, yet general and effective prompting method, Re2, to enhance the reasoning capabilities of off-the-shelf Large Language Models (LLMs) Unlike most thought-eliciting prompting methods, such as Chain-of-Thought (CoT), Re2 shifts the focus to the input by processing questions twice, thereby enhancing the understanding process. We evaluate Re2 on extensive reasoning benchmarks across 14 datasets, spanning 112 experiments, to validate its effectiveness and generality.
  arXiv  Detail & Related papers  (2023-09-12T14:36:23Z)
• Integrating Semantic Information into Sketchy Reading Module of Retro-Reader for Vietnamese Machine Reading Comprehension [0.22940141855172036]
  We use semantic role labels from the SRL task to add semantics to pre-trained language models such as mBERT, XLM-R, PhoBERT. The improved Retro-Reader model's encoder with semantics was first applied to the Vietnamese Machine Reading task and obtained positive results.
  arXiv  Detail & Related papers  (2023-01-01T15:28:27Z)
• Choose Your QA Model Wisely: A Systematic Study of Generative and Extractive Readers for Question Answering [41.14425610319543]
  We make the first attempt to systematically study the comparison of extractive and generative readers for question answering. To be aligned with the state-of-the-art, we explore nine transformer-based large pre-trained language models (PrLMs) as backbone architectures.
  arXiv  Detail & Related papers  (2022-03-14T22:07:52Z)
• Answer Span Correction in Machine Reading Comprehension [16.82391374339153]
  Machine reading comprehension (MRC) consists of verifying an extracted answer against an input context and question pair. Previous work has looked at re-assessing the "answerability" of the question given the extracted answer. Here we address the tendency of existing MRC systems to produce partially correct answers when presented with answerable questions.
  arXiv  Detail & Related papers  (2020-11-06T15:31:07Z)
• Composing Answer from Multi-spans for Reading Comprehension [77.32873012668783]
  We present a novel method to generate answers for non-extraction machine reading comprehension (MRC) tasks. The proposed method has a better performance on accurately generating long answers, and substantially outperforms two competitive typical one-span and Seq2Seq baseline decoders.
  arXiv  Detail & Related papers  (2020-09-14T01:44:42Z)
• Explicit Memory Tracker with Coarse-to-Fine Reasoning for Conversational Machine Reading [177.50355465392047]
  We present a new framework of conversational machine reading that comprises a novel Explicit Memory Tracker (EMT) Our framework generates clarification questions by adopting a coarse-to-fine reasoning strategy. EMT achieves new state-of-the-art results of 74.6% micro-averaged decision accuracy and 49.5 BLEU4.
  arXiv  Detail & Related papers  (2020-05-26T02:21:31Z)
• Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension [59.80926970481975]
  We study machine reading comprehension (MRC) on long texts. A model takes as inputs a lengthy document and a question and then extracts a text span from the document as an answer. We propose to let a model learn to chunk in a more flexible way via reinforcement learning.
  arXiv  Detail & Related papers  (2020-05-16T18:08:58Z)
• Knowledgeable Dialogue Reading Comprehension on Key Turns [84.1784903043884]
  Multi-choice machine reading comprehension (MRC) requires models to choose the correct answer from candidate options given a passage and a question. Our research focuses dialogue-based MRC, where the passages are multi-turn dialogues. It suffers from two challenges, the answer selection decision is made without support of latently helpful commonsense, and the multi-turn context may hide considerable irrelevant information.
  arXiv  Detail & Related papers  (2020-04-29T07:04:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.