Joint Contrastive Learning for Unsupervised Domain Adaptation
- URL: http://arxiv.org/abs/2006.10297v1
- Date: Thu, 18 Jun 2020 06:25:34 GMT
- Title: Joint Contrastive Learning for Unsupervised Domain Adaptation
- Authors: Changhwa Park, Jonghyun Lee, Jaeyoon Yoo, Minhoe Hur, Sungroh Yoon
- Abstract summary: We propose an alternative upper bound on the target error that explicitly considers the joint error to render it more manageable.
We introduce Joint Contrastive Learning to find class-level discriminative features, which is essential for minimizing the joint error.
Experiments on two real-world datasets demonstrate that JCL outperforms the state-of-the-art methods.
- Score: 20.799729748233343
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Enhancing feature transferability by matching marginal distributions has led
to improvements in domain adaptation, although this is at the expense of
feature discrimination. In particular, the ideal joint hypothesis error in the
target error upper bound, which was previously considered to be minute, has
been found to be significant, impairing its theoretical guarantee. In this
paper, we propose an alternative upper bound on the target error that
explicitly considers the joint error to render it more manageable. With the
theoretical analysis, we suggest a joint optimization framework that combines
the source and target domains. Further, we introduce Joint Contrastive Learning
(JCL) to find class-level discriminative features, which is essential for
minimizing the joint error. With a solid theoretical framework, JCL employs
contrastive loss to maximize the mutual information between a feature and its
label, which is equivalent to maximizing the Jensen-Shannon divergence between
conditional distributions. Experiments on two real-world datasets demonstrate
that JCL outperforms the state-of-the-art methods.
Related papers
- Theoretical Guarantees for Domain Adaptation with Hierarchical Optimal
Transport [0.0]
Domain adaptation arises as an important problem in statistical learning theory.
Recent advances show that the success of domain adaptation algorithms heavily relies on their ability to minimize the divergence between the probability distributions of the source and target domains.
We propose a new theoretical framework for domain adaptation through hierarchical optimal transport.
arXiv Detail & Related papers (2022-10-24T15:34:09Z) - Domain-Specific Risk Minimization for Out-of-Distribution Generalization [104.17683265084757]
We first establish a generalization bound that explicitly considers the adaptivity gap.
We propose effective gap estimation methods for guiding the selection of a better hypothesis for the target.
The other method is minimizing the gap directly by adapting model parameters using online target samples.
arXiv Detail & Related papers (2022-08-18T06:42:49Z) - Learning Unbiased Transferability for Domain Adaptation by Uncertainty
Modeling [107.24387363079629]
Domain adaptation aims to transfer knowledge from a labeled source domain to an unlabeled or a less labeled but related target domain.
Due to the imbalance between the amount of annotated data in the source and target domains, only the target distribution is aligned to the source domain.
We propose a non-intrusive Unbiased Transferability Estimation Plug-in (UTEP) by modeling the uncertainty of a discriminator in adversarial-based DA methods to optimize unbiased transfer.
arXiv Detail & Related papers (2022-06-02T21:58:54Z) - On Tilted Losses in Machine Learning: Theory and Applications [26.87656095874882]
Exponential tilting is a technique commonly used in fields such as statistics, probability, information theory, and optimization.
We study a simple extension to ERM, which uses exponential tilting to flexibly tune the impact of individual losses.
We find that the framework can consistently outperform ERM and deliver competitive performance with state-of-the-art, problem-specific approaches.
arXiv Detail & Related papers (2021-09-13T17:33:42Z) - Margin Preserving Self-paced Contrastive Learning Towards Domain
Adaptation for Medical Image Segmentation [51.93711960601973]
We propose a novel margin preserving self-paced contrastive Learning model for cross-modal medical image segmentation.
With the guidance of progressively refined semantic prototypes, a novel margin preserving contrastive loss is proposed to boost the discriminability of embedded representation space.
Experiments on cross-modal cardiac segmentation tasks demonstrate that MPSCL significantly improves semantic segmentation performance.
arXiv Detail & Related papers (2021-03-15T15:23:10Z) - A Unified Joint Maximum Mean Discrepancy for Domain Adaptation [73.44809425486767]
This paper theoretically derives a unified form of JMMD that is easy to optimize.
From the revealed unified JMMD, we illustrate that JMMD degrades the feature-label dependence that benefits to classification.
We propose a novel MMD matrix to promote the dependence, and devise a novel label kernel that is robust to label distribution shift.
arXiv Detail & Related papers (2021-01-25T09:46:14Z) - Learning Invariant Representations and Risks for Semi-supervised Domain
Adaptation [109.73983088432364]
We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA)
We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
arXiv Detail & Related papers (2020-10-09T15:42:35Z) - Beyond $\mathcal{H}$-Divergence: Domain Adaptation Theory With
Jensen-Shannon Divergence [21.295136514836788]
We reveal the incoherence between the widely-adopted empirical domain adversarial training and its generally-assumed theoretical counterpart based on $mathcalH$-divergence.
We establish a new theoretical framework by directly proving the upper and lower target risk bounds based on joint distributional Jensen-Shannon divergence.
arXiv Detail & Related papers (2020-07-30T16:19:59Z) - Rethink Maximum Mean Discrepancy for Domain Adaptation [77.2560592127872]
This paper theoretically proves two essential facts: 1) minimizing the Maximum Mean Discrepancy equals to maximize the source and target intra-class distances respectively but jointly minimize their variance with some implicit weights, so that the feature discriminability degrades.
Experiments on several benchmark datasets not only prove the validity of theoretical results but also demonstrate that our approach could perform better than the comparative state-of-art methods substantially.
arXiv Detail & Related papers (2020-07-01T18:25:10Z) - Self-adaptive Re-weighted Adversarial Domain Adaptation [12.73753413032972]
We present a self-adaptive re-weighted adversarial domain adaptation approach.
It tries to enhance domain alignment from the perspective of conditional distribution.
Empirical evidence demonstrates that the proposed model outperforms state of the arts on standard domain adaptation datasets.
arXiv Detail & Related papers (2020-05-30T08:35:18Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.