A Theory of Multiple-Source Adaptation with Limited Target Labeled Data

• URL: http://arxiv.org/abs/2007.09762v2
• Date: Thu, 29 Oct 2020 16:41:50 GMT
• Title: A Theory of Multiple-Source Adaptation with Limited Target Labeled Data
• Authors: Yishay Mansour and Mehryar Mohri and Jae Ro and Ananda Theertha Suresh and Ke Wu
• Abstract summary: We show that a new family of algorithms based on model selection ideas benefits from very favorable guarantees in this scenario. We also report the results of several experiments with our algorithms that demonstrate their practical effectiveness.
• Score: 66.53679520072978
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a theoretical and algorithmic study of the multiple-source domain adaptation problem in the common scenario where the learner has access only to a limited amount of labeled target data, but where the learner has at disposal a large amount of labeled data from multiple source domains. We show that a new family of algorithms based on model selection ideas benefits from very favorable guarantees in this scenario and discuss some theoretical obstacles affecting some alternative techniques. We also report the results of several experiments with our algorithms that demonstrate their practical effectiveness.

Related papers

• Simple Ingredients for Offline Reinforcement Learning [86.1988266277766]
  offline reinforcement learning algorithms have proven effective on datasets highly connected to the target downstream task. We show that existing methods struggle with diverse data: their performance considerably deteriorates as data collected for related but different tasks is simply added to the offline buffer. We show that scale, more than algorithmic considerations, is the key factor influencing performance.
  arXiv  Detail & Related papers  (2024-03-19T18:57:53Z)
• Prominent Roles of Conditionally Invariant Components in Domain Adaptation: Theory and Algorithms [10.949415951813661]
  Domain adaptation (DA) is a statistical learning problem that arises when the distribution of the source data used to train a model differs from that of the target data used to evaluate the model. We show that conditionally invariant components (CICs) are relevant for prediction and remain conditionally invariant across the source and target data. We propose a new algorithm based on CICs, importance-weighted conditional invariant penalty (IW-CIP), which has target risk guarantees beyond simple settings.
  arXiv  Detail & Related papers  (2023-09-19T04:04:59Z)
• Best-Effort Adaptation [62.00856290846247]
  We present a new theoretical analysis of sample reweighting methods, including bounds holding uniformly over the weights. We show how these bounds can guide the design of learning algorithms that we discuss in detail. We report the results of a series of experiments demonstrating the effectiveness of our best-effort adaptation and domain adaptation algorithms.
  arXiv  Detail & Related papers  (2023-05-10T00:09:07Z)
• On Balancing Bias and Variance in Unsupervised Multi-Source-Free Domain Adaptation [6.2200089460762085]
  Methods for multi-source-free domain adaptation (MSFDA) typically train a target model using pseudo-labeled data produced by the source models. We develop an information-theoretic bound on the generalization error of the resulting target model. We then provide insights on how to balance this trade-off from three perspectives, including domain aggregation, selective pseudo-labeling, and joint feature alignment.
  arXiv  Detail & Related papers  (2022-02-01T22:34:18Z)
• Unsupervised Multi-source Domain Adaptation Without Access to Source Data [58.551861130011886]
  Unsupervised Domain Adaptation (UDA) aims to learn a predictor model for an unlabeled domain by transferring knowledge from a separate labeled source domain. We propose a novel and efficient algorithm which automatically combines the source models with suitable weights in such a way that it performs at least as good as the best source model.
  arXiv  Detail & Related papers  (2021-04-05T10:45:12Z)
• A Review of Single-Source Deep Unsupervised Visual Domain Adaptation [81.07994783143533]
  Large-scale labeled training datasets have enabled deep neural networks to excel across a wide range of benchmark vision tasks. In many applications, it is prohibitively expensive and time-consuming to obtain large quantities of labeled data. To cope with limited labeled training data, many have attempted to directly apply models trained on a large-scale labeled source domain to another sparsely labeled or unlabeled target domain.
  arXiv  Detail & Related papers  (2020-09-01T00:06:50Z)
• A Discriminative Technique for Multiple-Source Adaptation [55.5865665284915]
  We present a new discriminative technique for the multiple-source adaptation, MSA, problem. Our solution only requires conditional probabilities that can easily be accurately estimated from unlabeled data from the source domains. Our experiments with real-world applications further demonstrate that our new discriminative MSA algorithm outperforms the previous generative solution.
  arXiv  Detail & Related papers  (2020-08-25T14:06:15Z)
• Unsupervised Multi-Class Domain Adaptation: Theory, Algorithms, and Practice [41.26481166771589]
  We study the formalism of unsupervised multi-class domain adaptation (multi-class UDA) We introduce a new algorithm of Domain-Symmetric Networks (SymmNets)
  arXiv  Detail & Related papers  (2020-02-20T11:26:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.