Improving Out-of-Distribution Robustness via Selective Augmentation

• URL: http://arxiv.org/abs/2201.00299v1
• Date: Sun, 2 Jan 2022 05:58:33 GMT
• Title: Improving Out-of-Distribution Robustness via Selective Augmentation
• Authors: Huaxiu Yao, Yu Wang, Sai Li, Linjun Zhang, Weixin Liang, James Zou, Chelsea Finn
• Abstract summary: Machine learning algorithms assume that training and test examples are drawn from the same distribution. distribution shift is a common problem in real-world applications and can cause models to perform dramatically worse at test time. We propose a mixup-based technique which learns invariant functions via selective augmentation called LISA.
• Score: 61.147630193060856
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Machine learning algorithms typically assume that training and test examples are drawn from the same distribution. However, distribution shift is a common problem in real-world applications and can cause models to perform dramatically worse at test time. In this paper, we specifically consider the problems of domain shifts and subpopulation shifts (eg. imbalanced data). While prior works often seek to explicitly regularize internal representations and predictors of the model to be domain invariant, we instead aim to regularize the whole function without restricting the model's internal representations. This leads to a simple mixup-based technique which learns invariant functions via selective augmentation called LISA. LISA selectively interpolates samples either with the same labels but different domains or with the same domain but different labels. We analyze a linear setting and theoretically show how LISA leads to a smaller worst-group error. Empirically, we study the effectiveness of LISA on nine benchmarks ranging from subpopulation shifts to domain shifts, and we find that LISA consistently outperforms other state-of-the-art methods.

Related papers

• SALUDA: Surface-based Automotive Lidar Unsupervised Domain Adaptation [62.889835139583965]
  We introduce an unsupervised auxiliary task of learning an implicit underlying surface representation simultaneously on source and target data. As both domains share the same latent representation, the model is forced to accommodate discrepancies between the two sources of data. Our experiments demonstrate that our method achieves a better performance than the current state of the art, both in real-to-real and synthetic-to-real scenarios.
  arXiv  Detail & Related papers  (2023-04-06T17:36:23Z)
• Distributional Shift Adaptation using Domain-Specific Features [41.91388601229745]
  In open-world scenarios, streaming big data can be Out-Of-Distribution (OOD) We propose a simple yet effective approach that relies on correlations in general regardless of whether the features are invariant or not. Our approach uses the most confidently predicted samples identified by an OOD base model to train a new model that effectively adapts to the target domain.
  arXiv  Detail & Related papers  (2022-11-09T04:16:21Z)
• Mapping conditional distributions for domain adaptation under generalized target shift [0.0]
  We consider the problem of unsupervised domain adaptation (UDA) between a source and a target domain under conditional and label shift a.k.a Generalized Target Shift (GeTarS) Recent approaches learn domain-invariant representations, yet they have practical limitations and rely on strong assumptions that may not hold in practice. In this paper, we explore a novel and general approach to align pretrained representations, which circumvents existing drawbacks.
  arXiv  Detail & Related papers  (2021-10-26T14:25:07Z)
• Learning Domain Invariant Representations by Joint Wasserstein Distance Minimization [3.382067152367334]
  Domain shifts in the training data are common in practical applications of machine learning. Ideally, a ML model should work well independently of these shifts, for example, by learning a domain-invariant representation. Common ML losses do not give strong guarantees on how consistently the ML model performs for different domains.
  arXiv  Detail & Related papers  (2021-06-09T09:08:51Z)
• SelfReg: Self-supervised Contrastive Regularization for Domain Generalization [7.512471799525974]
  We propose a new regularization method for domain generalization based on contrastive learning, self-supervised contrastive regularization (SelfReg) The proposed approach use only positive data pairs, thus it resolves various problems caused by negative pair sampling. In the recent benchmark, DomainBed, the proposed method shows comparable performance to the conventional state-of-the-art alternatives.
  arXiv  Detail & Related papers  (2021-04-20T09:08:29Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)
• Adaptive Risk Minimization: Learning to Adapt to Domain Shift [109.87561509436016]
  A fundamental assumption of most machine learning algorithms is that the training and test data are drawn from the same underlying distribution. In this work, we consider the problem setting of domain generalization, where the training data are structured into domains and there may be multiple test time shifts. We introduce the framework of adaptive risk minimization (ARM), in which models are directly optimized for effective adaptation to shift by learning to adapt on the training domains.
  arXiv  Detail & Related papers  (2020-07-06T17:59:30Z)
• Estimating Generalization under Distribution Shifts via Domain-Invariant Representations [75.74928159249225]
  We use a set of domain-invariant predictors as a proxy for the unknown, true target labels. The error of the resulting risk estimate depends on the target risk of the proxy model.
  arXiv  Detail & Related papers  (2020-07-06T17:21:24Z)
• Few-Shot Learning as Domain Adaptation: Algorithm and Analysis [120.75020271706978]
  Few-shot learning uses prior knowledge learned from the seen classes to recognize the unseen classes. This class-difference-caused distribution shift can be considered as a special case of domain shift. We propose a prototypical domain adaptation network with attention (DAPNA) to explicitly tackle such a domain shift problem in a meta-learning framework.
  arXiv  Detail & Related papers  (2020-02-06T01:04:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.