R2-D2: A Modular Baseline for Open-Domain Question Answering

• URL: http://arxiv.org/abs/2109.03502v1
• Date: Wed, 8 Sep 2021 08:51:27 GMT
• Title: R2-D2: A Modular Baseline for Open-Domain Question Answering
• Abstract summary: This work presents a novel four-stage open-domain QA pipeline R2-D2 (Rank twice, reaD twice). The pipeline is composed of a retriever, passage reranker, extractive reader, generative reader and a mechanism that aggregates the final prediction from all system's components.
• Score: 0.1529342790344802
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: This work presents a novel four-stage open-domain QA pipeline R2-D2 (Rank twice, reaD twice). The pipeline is composed of a retriever, passage reranker, extractive reader, generative reader and a mechanism that aggregates the final prediction from all system's components. We demonstrate its strength across three open-domain QA datasets: NaturalQuestions, TriviaQA and EfficientQA, surpassing state-of-the-art on the first two. Our analysis demonstrates that: (i) combining extractive and generative reader yields absolute improvements up to 5 exact match and it is at least twice as effective as the posterior averaging ensemble of the same models with different parameters, (ii) the extractive reader with fewer parameters can match the performance of the generative reader on extractive QA datasets.

Related papers

• TeaRAG: A Token-Efficient Agentic Retrieval-Augmented Generation Framework [62.66056331998838]
  TeaRAG is a token-efficient agentic RAG framework capable of compressing both retrieval content and reasoning steps.<n>Our reward function evaluates the knowledge sufficiency by a knowledge matching mechanism, while penalizing excessive reasoning steps.
  arXiv  Detail & Related papers  (2025-11-07T16:08:34Z)
• BRIEF: Bridging Retrieval and Inference for Multi-hop Reasoning via Compression [91.23933111083389]
  Retrieval-augmented generation (RAG) can supplement large language models (LLMs) by integrating external knowledge. This paper presents BRIEF, a lightweight approach that performs query-aware multi-hop reasoning. Based on our synthetic data built entirely by open-source models, BRIEF generates more concise summaries.
  arXiv  Detail & Related papers  (2024-10-20T04:24:16Z)
• A Unified One-Step Solution for Aspect Sentiment Quad Prediction [3.428123050377681]
  Aspect sentiment quad prediction (ASQP) is a challenging yet significant subtask in aspect-based sentiment analysis. We release two new datasets for ASQP, which contain the following characteristics: larger size, more words per sample, and higher density. We propose a unified one-step solution for ASQP, namely One-ASQP, to detect the aspect categories and to identify the aspect-opinion-sentiment triplets simultaneously.
  arXiv  Detail & Related papers  (2023-06-07T05:00:01Z)
• Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformer [80.50327229467993]
  We show that a single model trained end-to-end can achieve both competitive retrieval and QA performance. We show that end-to-end adaptation significantly boosts its performance on out-of-domain datasets in both supervised and unsupervised settings.
  arXiv  Detail & Related papers  (2022-12-05T04:51:21Z)
• Grape: Knowledge Graph Enhanced Passage Reader for Open-domain Question Answering [36.85435188308579]
  A common thread of open-domain question answering (QA) models employs a retriever-reader pipeline that first retrieves a handful of relevant passages from Wikipedia. We propose a novel knowledge Graph enhanced passage reader, namely Grape, to improve the reader performance for open-domain QA.
  arXiv  Detail & Related papers  (2022-10-06T14:06:04Z)
• KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering [68.00631278030627]
  We propose a novel method KG-FiD, which filters noisy passages by leveraging the structural relationship among the retrieved passages with a knowledge graph. We show that KG-FiD can improve vanilla FiD by up to 1.5% on answer exact match score and achieve comparable performance with FiD with only 40% of computation cost.
  arXiv  Detail & Related papers  (2021-10-08T18:39:59Z)
• Question Answering Infused Pre-training of General-Purpose Contextualized Representations [70.62967781515127]
  We propose a pre-training objective based on question answering (QA) for learning general-purpose contextual representations. We accomplish this goal by training a bi-encoder QA model, which independently encodes passages and questions, to match the predictions of a more accurate cross-encoder model. We show large improvements over both RoBERTa-large and previous state-of-the-art results on zero-shot and few-shot paraphrase detection.
  arXiv  Detail & Related papers  (2021-06-15T14:45:15Z)
• Data Augmentation for Abstractive Query-Focused Multi-Document Summarization [129.96147867496205]
  We present two QMDS training datasets, which we construct using two data augmentation methods. These two datasets have complementary properties, i.e., QMDSCNN has real summaries but queries are simulated, while QMDSIR has real queries but simulated summaries. We build end-to-end neural network models on the combined datasets that yield new state-of-the-art transfer results on DUC datasets.
  arXiv  Detail & Related papers  (2021-03-02T16:57:01Z)
• Pruning the Index Contents for Memory Efficient Open-Domain QA [0.1529342790344802]
  This work presents a novel pipeline that demonstrates what is achievable with a combined effort of state-of-the-art approaches. It proposes the novel R2-D2 pipeline composed of retriever, reranker, extractive reader, generative reader and a simple way to combine them. This work presents a simple approach for pruning the contents of a massive index such that the open-domain QA system altogether with index, OS, and library components fits into 6GiB docker image.
  arXiv  Detail & Related papers  (2021-02-21T21:56:38Z)
• End-to-End Training of Neural Retrievers for Open-Domain Question Answering [32.747113232867825]
  It remains unclear how unsupervised and supervised methods can be used most effectively for neural retrievers. We propose an approach of unsupervised pre-training with the Inverse Cloze Task and masked salient spans. We also explore two approaches for end-to-end supervised training of the reader and retriever components in OpenQA models.
  arXiv  Detail & Related papers  (2021-01-02T09:05:34Z)
• Open Question Answering over Tables and Text [55.8412170633547]
  In open question answering (QA), the answer to a question is produced by retrieving and then analyzing documents that might contain answers to the question. Most open QA systems have considered only retrieving information from unstructured text. We present a new large-scale dataset Open Table-and-Text Question Answering (OTT-QA) to evaluate performance on this task.
  arXiv  Detail & Related papers  (2020-10-20T16:48:14Z)
• Tradeoffs in Sentence Selection Techniques for Open-Domain Question Answering [54.541952928070344]
  We describe two groups of models for sentence selection: QA-based approaches, which run a full-fledged QA system to identify answer candidates, and retrieval-based models, which find parts of each passage specifically related to each question. We show that very lightweight QA models can do well at this task, but retrieval-based models are faster still.
  arXiv  Detail & Related papers  (2020-09-18T23:39:15Z)
• MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Models [25.398047573530985]
  Retrieval question answering (ReQA) is the task of retrieving a sentence-level answer to a question from an open corpus. This paper presents MultiReQA, anew multi-domain ReQA evaluation suite com-posed of eight retrieval QA tasks drawn from publicly available QA datasets.
  arXiv  Detail & Related papers  (2020-05-05T21:30:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.