Continual Domain Adversarial Adaptation via Double-Head Discriminators

• URL: http://arxiv.org/abs/2402.03588v1
• Date: Mon, 5 Feb 2024 23:46:03 GMT
• Title: Continual Domain Adversarial Adaptation via Double-Head Discriminators
• Authors: Yan Shen and Zhanghexuan Ji and Chunwei Ma and Mingchen Gao
• Abstract summary: Domain adversarial adaptation in a continual setting poses a significant challenge due to the limitations on accessing previous source domain data. We propose a double-head discriminator algorithm, by introducing an addition source-only domain discriminator. We prove that with the introduction of a pre-trained source-only domain discriminator, the empirical estimation error of $gH$-divergence related adversarial loss is reduced.
• Score: 9.27879320502565
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Domain adversarial adaptation in a continual setting poses a significant challenge due to the limitations on accessing previous source domain data. Despite extensive research in continual learning, the task of adversarial adaptation cannot be effectively accomplished using only a small number of stored source domain data, which is a standard setting in memory replay approaches. This limitation arises from the erroneous empirical estimation of $\gH$-divergence with few source domain samples. To tackle this problem, we propose a double-head discriminator algorithm, by introducing an addition source-only domain discriminator that are trained solely on source learning phase. We prove that with the introduction of a pre-trained source-only domain discriminator, the empirical estimation error of $\gH$-divergence related adversarial loss is reduced from the source domain side. Further experiments on existing domain adaptation benchmark show that our proposed algorithm achieves more than 2$\%$ improvement on all categories of target domain adaptation task while significantly mitigating the forgetting on source domain.

Related papers

• Contrastive Adversarial Training for Unsupervised Domain Adaptation [2.432037584128226]
  Domain adversarial training has been successfully adopted for various domain adaptation tasks. Large models make adversarial training being easily biased towards source domain and hardly adapted to target domain. We propose contrastive adversarial training (CAT) approach that leverages the labeled source domain samples to reinforce and regulate the feature generation for target domain.
  arXiv  Detail & Related papers  (2024-07-17T17:59:21Z)
• Bi-discriminator Domain Adversarial Neural Networks with Class-Level Gradient Alignment [87.8301166955305]
  We propose a novel bi-discriminator domain adversarial neural network with class-level gradient alignment. BACG resorts to gradient signals and second-order probability estimation for better alignment of domain distributions. In addition, inspired by contrastive learning, we develop a memory bank-based variant, i.e. Fast-BACG, which can greatly shorten the training process.
  arXiv  Detail & Related papers  (2023-10-21T09:53:17Z)
• IT-RUDA: Information Theory Assisted Robust Unsupervised Domain Adaptation [7.225445443960775]
  Distribution shift between train (source) and test (target) datasets is a common problem encountered in machine learning applications. UDA technique carries out knowledge transfer from a label-rich source domain to an unlabeled target domain. Outliers that exist in either source or target datasets can introduce additional challenges when using UDA in practice.
  arXiv  Detail & Related papers  (2022-10-24T04:33:52Z)
• Adversarial Bi-Regressor Network for Domain Adaptive Regression [52.5168835502987]
  It is essential to learn a cross-domain regressor to mitigate the domain shift. This paper proposes a novel method Adversarial Bi-Regressor Network (ABRNet) to seek more effective cross-domain regression model.
  arXiv  Detail & Related papers  (2022-09-20T18:38:28Z)
• Balancing Discriminability and Transferability for Source-Free Domain Adaptation [55.143687986324935]
  Conventional domain adaptation (DA) techniques aim to improve domain transferability by learning domain-invariant representations. The requirement of simultaneous access to labeled source and unlabeled target renders them unsuitable for the challenging source-free DA setting. We derive novel insights to show that a mixup between original and corresponding translated generic samples enhances the discriminability-transferability trade-off.
  arXiv  Detail & Related papers  (2022-06-16T09:06:22Z)
• Source-Free Domain Adaptation via Distribution Estimation [106.48277721860036]
  Domain Adaptation aims to transfer the knowledge learned from a labeled source domain to an unlabeled target domain whose data distributions are different. Recently, Source-Free Domain Adaptation (SFDA) has drawn much attention, which tries to tackle domain adaptation problem without using source data. In this work, we propose a novel framework called SFDA-DE to address SFDA task via source Distribution Estimation.
  arXiv  Detail & Related papers  (2022-04-24T12:22:19Z)
• Towards Fair Knowledge Transfer for Imbalanced Domain Adaptation [61.317911756566126]
  We propose a Towards Fair Knowledge Transfer framework to handle the fairness challenge in imbalanced cross-domain learning. Specifically, a novel cross-domain mixup generation is exploited to augment the minority source set with target information to enhance fairness. Our model significantly improves over 20% on two benchmarks in terms of the overall accuracy.
  arXiv  Detail & Related papers  (2020-10-23T06:29:09Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)
• Discrepancy Minimization in Domain Generalization with Generative Nearest Neighbors [13.047289562445242]
  Domain generalization (DG) deals with the problem of domain shift where a machine learning model trained on multiple-source domains fail to generalize well on a target domain with different statistics. Multiple approaches have been proposed to solve the problem of domain generalization by learning domain invariant representations across the source domains that fail to guarantee generalization on the shifted target domain. We propose a Generative Nearest Neighbor based Discrepancy Minimization (GNNDM) method which provides a theoretical guarantee that is upper bounded by the error in the labeling process of the target.
  arXiv  Detail & Related papers  (2020-07-28T14:54:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.