A Theory of Label Propagation for Subpopulation Shift
- URL: http://arxiv.org/abs/2102.11203v1
- Date: Mon, 22 Feb 2021 17:27:47 GMT
- Title: A Theory of Label Propagation for Subpopulation Shift
- Authors: Tianle Cai, Ruiqi Gao, Jason D. Lee, Qi Lei
- Abstract summary: We propose a provably effective framework for domain adaptation based on label propagation.
We obtain end-to-end finite-sample guarantees on the entire algorithm.
We extend our theoretical framework to a more general setting of source-to-target transfer based on a third unlabeled dataset.
- Score: 61.408438422417326
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: One of the central problems in machine learning is domain adaptation. Unlike
past theoretical work, we consider a new model for subpopulation shift in the
input or representation space. In this work, we propose a provably effective
framework for domain adaptation based on label propagation. In our analysis, we
use a simple but realistic ``expansion'' assumption, proposed in
\citet{wei2021theoretical}. Using a teacher classifier trained on the source
domain, our algorithm not only propagates to the target domain but also
improves upon the teacher. By leveraging existing generalization bounds, we
also obtain end-to-end finite-sample guarantees on the entire algorithm. In
addition, we extend our theoretical framework to a more general setting of
source-to-target transfer based on a third unlabeled dataset, which can be
easily applied in various learning scenarios.
Related papers
- Bidirectional Domain Mixup for Domain Adaptive Semantic Segmentation [73.3083304858763]
This paper systematically studies the impact of mixup under the domain adaptaive semantic segmentation task.
In specific, we achieve domain mixup in two-step: cut and paste.
We provide extensive ablation experiments to empirically verify our main components of the framework.
arXiv Detail & Related papers (2023-03-17T05:22:44Z) - Domain Adaptation via Rebalanced Sub-domain Alignment [22.68115322836635]
Unsupervised domain adaptation (UDA) is a technique used to transfer knowledge from a labeled source domain to a related unlabeled target domain.
Many UDA methods have shown success in the past, but they often assume that the source and target domains must have identical class label distributions.
We propose a novel generalization bound that reweights source classification error by aligning source and target sub-domains.
arXiv Detail & Related papers (2023-02-03T21:30:40Z) - Mapping conditional distributions for domain adaptation under
generalized target shift [0.0]
We consider the problem of unsupervised domain adaptation (UDA) between a source and a target domain under conditional and label shift a.k.a Generalized Target Shift (GeTarS)
Recent approaches learn domain-invariant representations, yet they have practical limitations and rely on strong assumptions that may not hold in practice.
In this paper, we explore a novel and general approach to align pretrained representations, which circumvents existing drawbacks.
arXiv Detail & Related papers (2021-10-26T14:25:07Z) - f-Domain-Adversarial Learning: Theory and Algorithms [82.97698406515667]
Unsupervised domain adaptation is used in many machine learning applications where, during training, a model has access to unlabeled data in the target domain.
We derive a novel generalization bound for domain adaptation that exploits a new measure of discrepancy between distributions based on a variational characterization of f-divergences.
arXiv Detail & Related papers (2021-06-21T18:21:09Z) - KL Guided Domain Adaptation [88.19298405363452]
Domain adaptation is an important problem and often needed for real-world applications.
A common approach in the domain adaptation literature is to learn a representation of the input that has the same distributions over the source and the target domain.
We show that with a probabilistic representation network, the KL term can be estimated efficiently via minibatch samples.
arXiv Detail & Related papers (2021-06-14T22:24:23Z) - On Universal Black-Box Domain Adaptation [53.7611757926922]
We study an arguably least restrictive setting of domain adaptation in a sense of practical deployment.
Only the interface of source model is available to the target domain, and where the label-space relations between the two domains are allowed to be different and unknown.
We propose to unify them into a self-training framework, regularized by consistency of predictions in local neighborhoods of target samples.
arXiv Detail & Related papers (2021-04-10T02:21:09Z) - Domain Adaptation in LiDAR Semantic Segmentation by Aligning Class
Distributions [9.581605678437032]
This work addresses the problem of unsupervised domain adaptation for LiDAR semantic segmentation models.
Our approach combines novel ideas on top of the current state-of-the-art approaches and yields new state-of-the-art results.
arXiv Detail & Related papers (2020-10-23T08:52:15Z) - Learning Invariant Representations and Risks for Semi-supervised Domain
Adaptation [109.73983088432364]
We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA)
We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
arXiv Detail & Related papers (2020-10-09T15:42:35Z) - Domain Adaptation by Class Centroid Matching and Local Manifold
Self-Learning [8.316259570013813]
We propose a novel domain adaptation approach, which can thoroughly explore the data distribution structure of target domain.
We regard the samples within the same cluster in target domain as a whole rather than individuals and assigns pseudo-labels to the target cluster by class centroid matching.
An efficient iterative optimization algorithm is designed to solve the objective function of our proposal with theoretical convergence guarantee.
arXiv Detail & Related papers (2020-03-20T16:59:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.