Frustratingly Hard Evidence Retrieval for QA Over Books

• URL: http://arxiv.org/abs/2007.09878v1
• Date: Mon, 20 Jul 2020 04:10:08 GMT
• Title: Frustratingly Hard Evidence Retrieval for QA Over Books
• Authors: Xiangyang Mou, Mo Yu, Bingsheng Yao, Chenghao Yang, Xiaoxiao Guo, Saloni Potdar, Hui Su
• Abstract summary: We formulate BookQA as an open-domain QA task given its similar dependency on evidence retrieval. We investigate how state-of-the-art open-domain QA approaches can help BookQA.
• Score: 42.451721693007954
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A lot of progress has been made to improve question answering (QA) in recent years, but the special problem of QA over narrative book stories has not been explored in-depth. We formulate BookQA as an open-domain QA task given its similar dependency on evidence retrieval. We further investigate how state-of-the-art open-domain QA approaches can help BookQA. Besides achieving state-of-the-art on the NarrativeQA benchmark, our study also reveals the difficulty of evidence retrieval in books with a wealth of experiments and analysis - which necessitates future effort on novel solutions for evidence retrieval in BookQA.

Related papers

• Improving Health Question Answering with Reliable and Time-Aware Evidence Retrieval [5.69361786082969]
  Our study focuses on the open-domain QA setting, where the key challenge is to first uncover relevant evidence in large knowledge bases. By utilizing the common retrieve-then-read QA pipeline and PubMed as a trustworthy collection of medical research documents, we answer health questions from three diverse datasets. Our results reveal that cutting down on the amount of retrieved documents and favoring more recent and highly cited documents can improve the final macro F1 score up to 10%.
  arXiv  Detail & Related papers  (2024-04-12T09:56:12Z)
• Closed-book Question Generation via Contrastive Learning [20.644215991166895]
  We propose a new QG model empowered by a contrastive learning module and an answer reconstruction module. We show how to leverage the proposed model to improve existing closed-book QA systems.
  arXiv  Detail & Related papers  (2022-10-13T06:45:46Z)
• Can Question Rewriting Help Conversational Question Answering? [13.484873786389471]
  Question rewriting (QR) is a subtask of conversational question answering (CQA) We investigate a reinforcement learning approach that integrates QR and CQA tasks and does not require corresponding QR datasets for targeted CQA. We find, however, that the RL method is on par with the end-to-end baseline.
  arXiv  Detail & Related papers  (2022-04-13T08:16:03Z)
• Multifaceted Improvements for Conversational Open-Domain Question Answering [54.913313912927045]
  We propose a framework with Multifaceted Improvements for Conversational open-domain Question Answering (MICQA) Firstly, the proposed KL-divergence based regularization is able to lead to a better question understanding for retrieval and answer reading. Second, the added post-ranker module can push more relevant passages to the top placements and be selected for reader with a two-aspect constrains. Third, the well designed curriculum learning strategy effectively narrows the gap between the golden passage settings of training and inference, and encourages the reader to find true answer without the golden passage assistance.
  arXiv  Detail & Related papers  (2022-04-01T07:54:27Z)
• Narrative Question Answering with Cutting-Edge Open-Domain QA Techniques: A Comprehensive Study [45.9120218818558]
  We benchmark the research on the NarrativeQA dataset with experiments with cutting-edge ODQA techniques. This quantifies the challenges Book QA poses, as well as advances the published state-of-the-art with a $sim$7% absolute improvement on Rouge-L. Our findings indicate that the event-centric questions dominate this task, which exemplifies the inability of existing QA models to handle event-oriented scenarios.
  arXiv  Detail & Related papers  (2021-06-07T17:46:09Z)
• Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering [62.88322725956294]
  We review the latest research trends in OpenQA, with particular attention to systems that incorporate neural MRC techniques. We introduce modern OpenQA architecture named Retriever-Reader'' and analyze the various systems that follow this architecture. We then discuss key challenges to developing OpenQA systems and offer an analysis of benchmarks that are commonly used.
  arXiv  Detail & Related papers  (2021-01-04T04:47:46Z)
• Technical Question Answering across Tasks and Domains [47.80330046038137]
  We present an adjustable joint learning approach for document retrieval and reading comprehension tasks. Our experiments on the TechQA demonstrates superior performance compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-10-19T18:39:30Z)
• Harvesting and Refining Question-Answer Pairs for Unsupervised QA [95.9105154311491]
  We introduce two approaches to improve unsupervised Question Answering (QA) First, we harvest lexically and syntactically divergent questions from Wikipedia to automatically construct a corpus of question-answer pairs (named as RefQA) Second, we take advantage of the QA model to extract more appropriate answers, which iteratively refines data over RefQA.
  arXiv  Detail & Related papers  (2020-05-06T15:56:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.