Synthetic Target Domain Supervision for Open Retrieval QA

• URL: http://arxiv.org/abs/2204.09248v1
• Date: Wed, 20 Apr 2022 06:28:13 GMT
• Title: Synthetic Target Domain Supervision for Open Retrieval QA
• Authors: Revanth Gangi Reddy, Bhavani Iyer, Md Arafat Sultan, Rong Zhang, Avirup Sil, Vittorio Castelli, Radu Florian, Salim Roukos
• Abstract summary: We stress-test the Dense Passage Retriever (DPR) on closed and specialized target domains such as COVID-19. DPR lags behind standard BM25 in this important real-world setting. In experiments, this noisy but fully automated target domain supervision gives DPR a sizable advantage over BM25.
• Score: 24.48364368847857
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural passage retrieval is a new and promising approach in open retrieval question answering. In this work, we stress-test the Dense Passage Retriever (DPR) -- a state-of-the-art (SOTA) open domain neural retrieval model -- on closed and specialized target domains such as COVID-19, and find that it lags behind standard BM25 in this important real-world setting. To make DPR more robust under domain shift, we explore its fine-tuning with synthetic training examples, which we generate from unlabeled target domain text using a text-to-text generator. In our experiments, this noisy but fully automated target domain supervision gives DPR a sizable advantage over BM25 in out-of-domain settings, making it a more viable model in practice. Finally, an ensemble of BM25 and our improved DPR model yields the best results, further pushing the SOTA for open retrieval QA on multiple out-of-domain test sets.

Related papers

• Quality > Quantity: Synthetic Corpora from Foundation Models for Closed-Domain Extractive Question Answering [35.38140071573828]
  We study extractive question answering within closed domains and introduce the concept of targeted pre-training. Our proposed framework uses Galactica to generate synthetic, targeted'' corpora that align with specific writing styles and topics.
  arXiv  Detail & Related papers  (2023-10-25T20:48:16Z)
• Self-training through Classifier Disagreement for Cross-Domain Opinion Target Extraction [62.41511766918932]
  Opinion target extraction (OTE) or aspect extraction (AE) is a fundamental task in opinion mining. Recent work focus on cross-domain OTE, which is typically encountered in real-world scenarios. We propose a new SSL approach that opts for selecting target samples whose model output from a domain-specific teacher and student network disagrees on the unlabelled target data.
  arXiv  Detail & Related papers  (2023-02-28T16:31:17Z)
• To Adapt or to Annotate: Challenges and Interventions for Domain Adaptation in Open-Domain Question Answering [46.403929561360485]
  We study end-to-end model performance of open-domain question answering (ODQA) We find that not only do models fail to generalize, but high retrieval scores often still yield poor answer prediction accuracy. We propose and evaluate several intervention methods which improve end-to-end answer F1 score by up to 24 points.
  arXiv  Detail & Related papers  (2022-12-20T16:06:09Z)
• Domain-Agnostic Prior for Transfer Semantic Segmentation [197.9378107222422]
  Unsupervised domain adaptation (UDA) is an important topic in the computer vision community. We present a mechanism that regularizes cross-domain representation learning with a domain-agnostic prior (DAP) Our research reveals that UDA benefits much from better proxies, possibly from other data modalities.
  arXiv  Detail & Related papers  (2022-04-06T09:13:25Z)
• Amplitude Spectrum Transformation for Open Compound Domain Adaptive Semantic Segmentation [62.68759523116924]
  Open compound domain adaptation (OCDA) has emerged as a practical adaptation setting. We propose a novel feature space Amplitude Spectrum Transformation (AST)
  arXiv  Detail & Related papers  (2022-02-09T05:40:34Z)
• Unsupervised Domain Generalization for Person Re-identification: A Domain-specific Adaptive Framework [50.88463458896428]
  Domain generalization (DG) has attracted much attention in person re-identification (ReID) recently. Existing methods usually need the source domains to be labeled, which could be a significant burden for practical ReID tasks. We propose a simple and efficient domain-specific adaptive framework, and realize it with an adaptive normalization module.
  arXiv  Detail & Related papers  (2021-11-30T02:35:51Z)
• Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant Representations [24.703464680511154]
  We propose Momentum adversarial Domain Invariant Representation learning (MoDIR) MoDIR trains a domain classifier distinguishing source versus target, and then adversarially updates the DR encoder to learn domain invariant representations. Our experiments show that MoDIR robustly outperforms its baselines on 10+ ranking datasets from the BEIR benchmark in the zero-shot setup.
  arXiv  Detail & Related papers  (2021-10-14T17:45:06Z)
• Discover, Hallucinate, and Adapt: Open Compound Domain Adaptation for Semantic Segmentation [91.30558794056056]
  Unsupervised domain adaptation (UDA) for semantic segmentation has been attracting attention recently. We present a novel framework based on three main design principles: discover, hallucinate, and adapt. We evaluate our solution on standard benchmark GTA to C-driving, and achieved new state-of-the-art results.
  arXiv  Detail & Related papers  (2021-10-08T13:20:09Z)
• End-to-End QA on COVID-19: Domain Adaptation with Synthetic Training [13.731352294133211]
  End-to-end question answering requires both information retrieval and machine reading comprehension. Recent work has successfully trained neural IR systems using only supervised question answering (QA) examples from open-domain datasets. We combine our neural IR and MRC systems and show significant improvements in end-to-end QA on the CORD-19 collection.
  arXiv  Detail & Related papers  (2020-12-02T18:59:59Z)
• Dense Passage Retrieval for Open-Domain Question Answering [49.028342823838486]
  We show that retrieval can be practically implemented using dense representations alone. Our dense retriever outperforms a strong Lucene-BM25 system largely by 9%-19% absolute in terms of top-20 passage retrieval accuracy.
  arXiv  Detail & Related papers  (2020-04-10T04:53:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.