Few-shot Domain Adaptation by Causal Mechanism Transfer

• URL: http://arxiv.org/abs/2002.03497v2
• Date: Wed, 19 Aug 2020 00:10:29 GMT
• Title: Few-shot Domain Adaptation by Causal Mechanism Transfer
• Authors: Takeshi Teshima, Issei Sato, Masashi Sugiyama
• Abstract summary: We study few-shot supervised domain adaptation (DA) for regression problems, where only a few labeled target domain data and many labeled source domain data are available. Many of the current DA methods base their transfer assumptions on either parametrized distribution shift or apparent distribution similarities. We propose mechanism transfer, a meta-distributional scenario in which a data generating mechanism is invariant among domains.
• Score: 107.08605582020866
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study few-shot supervised domain adaptation (DA) for regression problems, where only a few labeled target domain data and many labeled source domain data are available. Many of the current DA methods base their transfer assumptions on either parametrized distribution shift or apparent distribution similarities, e.g., identical conditionals or small distributional discrepancies. However, these assumptions may preclude the possibility of adaptation from intricately shifted and apparently very different distributions. To overcome this problem, we propose mechanism transfer, a meta-distributional scenario in which a data generating mechanism is invariant among domains. This transfer assumption can accommodate nonparametric shifts resulting in apparently different distributions while providing a solid statistical basis for DA. We take the structural equations in causal modeling as an example and propose a novel DA method, which is shown to be useful both theoretically and experimentally. Our method can be seen as the first attempt to fully leverage the structural causal models for DA.

Related papers

• Domain Agnostic Conditional Invariant Predictions for Domain Generalization [20.964740750976667]
  We propose a Discriminant Risk Minimization (DRM) theory and the corresponding algorithm to capture the invariant features without domain labels. In DRM theory, we prove that reducing the discrepancy of prediction distribution between overall source domain and any subset of it can contribute to obtaining invariant features. We evaluate our algorithm against various domain generalization methods on multiple real-world datasets.
  arXiv  Detail & Related papers  (2024-06-09T02:38:52Z)
• Proxy Methods for Domain Adaptation [78.03254010884783]
  proxy variables allow for adaptation to distribution shift without explicitly recovering or modeling latent variables. We develop a two-stage kernel estimation approach to adapt to complex distribution shifts in both settings.
  arXiv  Detail & Related papers  (2024-03-12T09:32:41Z)
• Unsupervised Domain Adaptation Based on the Predictive Uncertainty of Models [1.6498361958317636]
  Unsupervised domain adaptation (UDA) aims to improve the prediction performance in the target domain under distribution shifts from the source domain. We present a novel UDA method that learns domain-invariant features that minimize the domain divergence.
  arXiv  Detail & Related papers  (2022-11-16T12:23:32Z)
• Super-model ecosystem: A domain-adaptation perspective [101.76769818069072]
  This paper attempts to establish the theoretical foundation for the emerging super-model paradigm via domain adaptation. Super-model paradigms help reduce computational and data cost and carbon emission, which is critical to AI industry.
  arXiv  Detail & Related papers  (2022-08-30T09:09:43Z)
• Domain-Specific Risk Minimization for Out-of-Distribution Generalization [104.17683265084757]
  We first establish a generalization bound that explicitly considers the adaptivity gap. We propose effective gap estimation methods for guiding the selection of a better hypothesis for the target. The other method is minimizing the gap directly by adapting model parameters using online target samples.
  arXiv  Detail & Related papers  (2022-08-18T06:42:49Z)
• Causal Discovery in Heterogeneous Environments Under the Sparse Mechanism Shift Hypothesis [7.895866278697778]
  Machine learning approaches commonly rely on the assumption of independent and identically distributed (i.i.d.) data. In reality, this assumption is almost always violated due to distribution shifts between environments. We propose the Mechanism Shift Score (MSS), a score-based approach amenable to various empirical estimators.
  arXiv  Detail & Related papers  (2022-06-04T15:39:30Z)
• Instrumental Variable-Driven Domain Generalization with Unobserved Confounders [53.735614014067394]
  Domain generalization (DG) aims to learn from multiple source domains a model that can generalize well on unseen target domains. We propose an instrumental variable-driven DG method (IV-DG) by removing the bias of the unobserved confounders with two-stage learning. In the first stage, it learns the conditional distribution of the input features of one domain given input features of another domain. In the second stage, it estimates the relationship by predicting labels with the learned conditional distribution.
  arXiv  Detail & Related papers  (2021-10-04T13:32:57Z)
• Domain Conditional Predictors for Domain Adaptation [3.951376400628575]
  We consider a conditional modeling approach in which predictions, in addition to being dependent on the input data, use information relative to the underlying data-generating distribution. We argue that such an approach is more generally applicable than current domain adaptation methods.
  arXiv  Detail & Related papers  (2021-06-25T22:15:54Z)
• Contrastive ACE: Domain Generalization Through Alignment of Causal Mechanisms [34.99779761100095]
  Domain generalization aims to learn knowledge invariant across different distributions. We consider the causal invariance of the average causal effect of the features to the labels.
  arXiv  Detail & Related papers  (2021-06-02T04:01:22Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.