Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant Representations

• URL: http://arxiv.org/abs/2110.07581v1
• Date: Thu, 14 Oct 2021 17:45:06 GMT
• Title: Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant Representations
• Authors: Ji Xin, Chenyan Xiong, Ashwin Srinivasan, Ankita Sharma, Damien Jose, Paul N. Bennett
• Abstract summary: We propose Momentum adversarial Domain Invariant Representation learning (MoDIR) MoDIR trains a domain classifier distinguishing source versus target, and then adversarially updates the DR encoder to learn domain invariant representations. Our experiments show that MoDIR robustly outperforms its baselines on 10+ ranking datasets from the BEIR benchmark in the zero-shot setup.
• Score: 24.703464680511154
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dense retrieval (DR) methods conduct text retrieval by first encoding texts in the embedding space and then matching them by nearest neighbor search. This requires strong locality properties from the representation space, i.e, the close allocations of each small group of relevant texts, which are hard to generalize to domains without sufficient training data. In this paper, we aim to improve the generalization ability of DR models from source training domains with rich supervision signals to target domains without any relevant labels, in the zero-shot setting. To achieve that, we propose Momentum adversarial Domain Invariant Representation learning (MoDIR), which introduces a momentum method in the DR training process to train a domain classifier distinguishing source versus target, and then adversarially updates the DR encoder to learn domain invariant representations. Our experiments show that MoDIR robustly outperforms its baselines on 10+ ranking datasets from the BEIR benchmark in the zero-shot setup, with more than 10% relative gains on datasets with enough sensitivity for DR models' evaluation. Source code of this paper will be released.

Related papers

• Remining Hard Negatives for Generative Pseudo Labeled Domain Adaptation [0.649970685896541]
  A state-of-the-art domain adaptation technique is Generative Pseudo Labeling (GPL) We analyze the documents retrieved by the domain-adapted model and discover that these are more relevant to the target queries than those of the non-adapted model. Our remining R-GPL approach boosts ranking performance in 13/14 BEIR datasets and 9/12 LoTTe datasets.
  arXiv  Detail & Related papers  (2025-01-24T12:02:37Z)
• ZoRI: Towards Discriminative Zero-Shot Remote Sensing Instance Segmentation [23.40908829241552]
  We propose a novel task called zero-shot remote sensing instance segmentation, aimed at identifying aerial objects that are absent from training data. We introduce a knowledge-injected adaptation strategy that decouples semantic-related information to preserve the pretrained vision-language alignment. We establish new experimental protocols and benchmarks, and extensive experiments convincingly demonstrate that ZoRI achieves the state-of-art performance.
  arXiv  Detail & Related papers  (2024-12-17T11:00:56Z)
• Heterogeneous Graph-based Framework with Disentangled Representations Learning for Multi-target Cross Domain Recommendation [7.247438542823219]
  CDR (Cross-Domain Recommendation) is a critical solution to data sparsity problem in recommendation system. We present HGDR (Heterogeneous Graph-based Framework with Disentangled Representations Learning), an end-to-end heterogeneous network architecture. Experiments on real-world datasets and online A/B tests prove that our proposed model can transmit information among domains effectively.
  arXiv  Detail & Related papers  (2024-07-01T02:27:54Z)
• Explaining Cross-Domain Recognition with Interpretable Deep Classifier [100.63114424262234]
  Interpretable Deep (IDC) learns the nearest source samples of a target sample as evidence upon which the classifier makes the decision. Our IDC leads to a more explainable model with almost no accuracy degradation and effectively calibrates classification for optimum reject options.
  arXiv  Detail & Related papers  (2022-11-15T15:58:56Z)
• Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning [122.62311703151215]
  Divide and Contrast (DaC) aims to connect the good ends of both worlds while bypassing their limitations. DaC divides the target data into source-like and target-specific samples, where either group of samples is treated with tailored goals. We further align the source-like domain with the target-specific samples using a memory bank-based Maximum Mean Discrepancy (MMD) loss to reduce the distribution mismatch.
  arXiv  Detail & Related papers  (2022-11-12T09:21:49Z)
• Disentangled Modeling of Domain and Relevance for Adaptable Dense Retrieval [54.349418995689284]
  We propose a novel Dense Retrieval (DR) framework named Disentangled Dense Retrieval ( DDR) to support effective domain adaptation for DR models. By making the REM and DAMs disentangled, DDR enables a flexible training paradigm in which REM is trained with supervision once and DAMs are trained with unsupervised data. DDR significantly improves ranking performance compared to strong DR baselines and substantially outperforms traditional retrieval methods in most scenarios.
  arXiv  Detail & Related papers  (2022-08-11T11:18:50Z)
• Domain-Invariant Proposals based on a Balanced Domain Classifier for Object Detection [8.583307102907295]
  Object recognition from images means to automatically find object(s) of interest and to return their category and location information. Benefiting from research on deep learning, like convolutional neural networks(CNNs) and generative adversarial networks, the performance in this field has been improved significantly. mismatching distributions, i.e., domain shifts, lead to a significant performance drop.
  arXiv  Detail & Related papers  (2022-02-12T00:21:27Z)
• Low-confidence Samples Matter for Domain Adaptation [47.552605279925736]
  Domain adaptation (DA) aims to transfer knowledge from a label-rich source domain to a related but label-scarce target domain. We propose a novel contrastive learning method by processing low-confidence samples. We evaluate the proposed method in both unsupervised and semi-supervised DA settings.
  arXiv  Detail & Related papers  (2022-02-06T15:45:45Z)
• Stagewise Unsupervised Domain Adaptation with Adversarial Self-Training for Road Segmentation of Remote Sensing Images [93.50240389540252]
  Road segmentation from remote sensing images is a challenging task with wide ranges of application potentials. We propose a novel stagewise domain adaptation model called RoadDA to address the domain shift (DS) issue in this field. Experiment results on two benchmarks demonstrate that RoadDA can efficiently reduce the domain gap and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-28T09:29:14Z)
• Inferring Latent Domains for Unsupervised Deep Domain Adaptation [54.963823285456925]
  Unsupervised Domain Adaptation (UDA) refers to the problem of learning a model in a target domain where labeled data are not available. This paper introduces a novel deep architecture which addresses the problem of UDA by automatically discovering latent domains in visual datasets. We evaluate our approach on publicly available benchmarks, showing that it outperforms state-of-the-art domain adaptation methods.
  arXiv  Detail & Related papers  (2021-03-25T14:33:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.