RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering

• URL: http://arxiv.org/abs/2010.08191v2
• Date: Wed, 12 May 2021 07:52:32 GMT
• Title: RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering
• Authors: Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxiang Dong, Hua Wu, Haifeng Wang
• Abstract summary: In open-domain question answering, dense passage retrieval has become a new paradigm to retrieve relevant passages for finding answers. We propose an optimized training approach, called RocketQA, to improve dense passage retrieval. We make three major technical contributions in RocketQA, namely cross-batch negatives, denoised hard negatives and data augmentation.
• Score: 55.280108297460636
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In open-domain question answering, dense passage retrieval has become a new paradigm to retrieve relevant passages for finding answers. Typically, the dual-encoder architecture is adopted to learn dense representations of questions and passages for semantic matching. However, it is difficult to effectively train a dual-encoder due to the challenges including the discrepancy between training and inference, the existence of unlabeled positives and limited training data. To address these challenges, we propose an optimized training approach, called RocketQA, to improving dense passage retrieval. We make three major technical contributions in RocketQA, namely cross-batch negatives, denoised hard negatives and data augmentation. The experiment results show that RocketQA significantly outperforms previous state-of-the-art models on both MSMARCO and Natural Questions. We also conduct extensive experiments to examine the effectiveness of the three strategies in RocketQA. Besides, we demonstrate that the performance of end-to-end QA can be improved based on our RocketQA retriever.

Related papers

• Towards Robust Extractive Question Answering Models: Rethinking the Training Methodology [0.34530027457862006]
  Previous research has shown that existing models, when trained on EQA datasets that include unanswerable questions, demonstrate a significant lack of robustness. Our proposed training method includes a novel loss function for the EQA problem and challenges an implicit assumption present in numerous EQA datasets. Our models exhibit significantly enhanced robustness against two types of adversarial attacks, with a performance decrease of only about a third compared to the default models.
  arXiv  Detail & Related papers  (2024-09-29T20:35:57Z)
• Query2Triple: Unified Query Encoding for Answering Diverse Complex Queries over Knowledge Graphs [29.863085746761556]
  We propose Query to Triple (Q2T), a novel approach that decouples the training for simple and complex queries. Our proposed Q2T is not only efficient to train, but also modular, thus easily adaptable to various neural link predictors.
  arXiv  Detail & Related papers  (2023-10-17T13:13:30Z)
• DQ-LoRe: Dual Queries with Low Rank Approximation Re-ranking for In-Context Learning [66.85379279041128]
  In this study, we introduce a framework that leverages Dual Queries and Low-rank approximation Re-ranking to automatically select exemplars for in-context learning. DQ-LoRe significantly outperforms prior state-of-the-art methods in the automatic selection of exemplars for GPT-4, enhancing performance from 92.5% to 94.2%.
  arXiv  Detail & Related papers  (2023-10-04T16:44:37Z)
• Modeling Multi-hop Question Answering as Single Sequence Prediction [88.72621430714985]
  We propose a simple generative approach (PathFid) that extends the task beyond just answer generation. PathFid explicitly models the reasoning process to resolve the answer for multi-hop questions. Our experiments demonstrate that PathFid leads to strong performance gains on two multi-hop QA datasets.
  arXiv  Detail & Related papers  (2022-05-18T21:57:59Z)
• RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking [89.82301733609279]
  We propose a novel joint training approach for dense passage retrieval and passage re-ranking. A major contribution is that we introduce the dynamic listwise distillation, where we design a unified listwise training approach for both the retriever and the re-ranker. During the dynamic distillation, the retriever and the re-ranker can be adaptively improved according to each other's relevance information.
  arXiv  Detail & Related papers  (2021-10-14T13:52:55Z)
• Learning to Rank Question Answer Pairs with Bilateral Contrastive Data Augmentation [39.22166065525888]
  We propose a novel and easy-to-apply data augmentation strategy, namely Bilateral Generation (BiG) With the augmented dataset, we design a contrastive training objective for learning to rank question answer pairs. Experimental results on three benchmark datasets, namely TREC-QA, WikiQA, and ANTIQUE, show that our method significantly improves the performance of ranking models.
  arXiv  Detail & Related papers  (2021-06-21T13:29:43Z)
• Harvesting and Refining Question-Answer Pairs for Unsupervised QA [95.9105154311491]
  We introduce two approaches to improve unsupervised Question Answering (QA) First, we harvest lexically and syntactically divergent questions from Wikipedia to automatically construct a corpus of question-answer pairs (named as RefQA) Second, we take advantage of the QA model to extract more appropriate answers, which iteratively refines data over RefQA.
  arXiv  Detail & Related papers  (2020-05-06T15:56:06Z)
• Learning to Recover Reasoning Chains for Multi-Hop Question Answering via Cooperative Games [66.98855910291292]
  We propose a new problem of learning to recover reasoning chains from weakly supervised signals. How the evidence passages are selected and how the selected passages are connected are handled by two models. For evaluation, we created benchmarks based on two multi-hop QA datasets.
  arXiv  Detail & Related papers  (2020-04-06T03:54:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.