Instrumental Variable-Driven Domain Generalization with Unobserved Confounders

• URL: http://arxiv.org/abs/2110.01438v2
• Date: Thu, 25 May 2023 08:21:33 GMT
• Title: Instrumental Variable-Driven Domain Generalization with Unobserved Confounders
• Authors: Junkun Yuan, Xu Ma, Ruoxuan Xiong, Mingming Gong, Xiangyu Liu, Fei Wu, Lanfen Lin, Kun Kuang
• Abstract summary: Domain generalization (DG) aims to learn from multiple source domains a model that can generalize well on unseen target domains. We propose an instrumental variable-driven DG method (IV-DG) by removing the bias of the unobserved confounders with two-stage learning. In the first stage, it learns the conditional distribution of the input features of one domain given input features of another domain. In the second stage, it estimates the relationship by predicting labels with the learned conditional distribution.
• Score: 53.735614014067394
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Domain generalization (DG) aims to learn from multiple source domains a model that can generalize well on unseen target domains. Existing DG methods mainly learn the representations with invariant marginal distribution of the input features, however, the invariance of the conditional distribution of the labels given the input features is more essential for unknown domain prediction. Meanwhile, the existing of unobserved confounders which affect the input features and labels simultaneously cause spurious correlation and hinder the learning of the invariant relationship contained in the conditional distribution. Interestingly, with a causal view on the data generating process, we find that the input features of one domain are valid instrumental variables for other domains. Inspired by this finding, we propose an instrumental variable-driven DG method (IV-DG) by removing the bias of the unobserved confounders with two-stage learning. In the first stage, it learns the conditional distribution of the input features of one domain given input features of another domain. In the second stage, it estimates the relationship by predicting labels with the learned conditional distribution. Theoretical analyses and simulation experiments show that it accurately captures the invariant relationship. Extensive experiments on real-world datasets demonstrate that IV-DG method yields state-of-the-art results.

Related papers

• Graphs Generalization under Distribution Shifts [11.963958151023732]
  We introduce a novel framework, namely Graph Learning Invariant Domain genERation (GLIDER) Our model outperforms baseline methods on node-level OOD generalization across domains in distribution shift on node features and topological structures simultaneously.
  arXiv  Detail & Related papers  (2024-03-25T00:15:34Z)
• Uncertainty-guided Contrastive Learning for Single Source Domain Generalisation [15.907643838530655]
  In this paper, we introduce a novel model referred to as Contrastive Uncertainty Domain Generalisation Network (CUDGNet) The key idea is to augment the source capacity in both input and label spaces through the fictitious domain generator. Our method also provides efficient uncertainty estimation at inference time from a single forward pass through the generator subnetwork.
  arXiv  Detail & Related papers  (2024-03-12T10:47:45Z)
• Algorithmic Fairness Generalization under Covariate and Dependence Shifts Simultaneously [28.24666589680547]
  We introduce a simple but effective approach that aims to learn a fair and invariant classifier. By augmenting various synthetic data domains through the model, we learn a fair and invariant classifier in source domains. This classifier can then be generalized to unknown target domains, maintaining both model prediction and fairness concerns.
  arXiv  Detail & Related papers  (2023-11-23T05:52:00Z)
• Improving Domain Generalization with Domain Relations [77.63345406973097]
  This paper focuses on domain shifts, which occur when the model is applied to new domains that are different from the ones it was trained on. We propose a new approach called D$3$G to learn domain-specific models. Our results show that D$3$G consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-02-06T08:11:16Z)
• Learning Unbiased Transferability for Domain Adaptation by Uncertainty Modeling [107.24387363079629]
  Domain adaptation aims to transfer knowledge from a labeled source domain to an unlabeled or a less labeled but related target domain. Due to the imbalance between the amount of annotated data in the source and target domains, only the target distribution is aligned to the source domain. We propose a non-intrusive Unbiased Transferability Estimation Plug-in (UTEP) by modeling the uncertainty of a discriminator in adversarial-based DA methods to optimize unbiased transfer.
  arXiv  Detail & Related papers  (2022-06-02T21:58:54Z)
• Contrastive ACE: Domain Generalization Through Alignment of Causal Mechanisms [34.99779761100095]
  Domain generalization aims to learn knowledge invariant across different distributions. We consider the causal invariance of the average causal effect of the features to the labels.
  arXiv  Detail & Related papers  (2021-06-02T04:01:22Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)
• Adaptively-Accumulated Knowledge Transfer for Partial Domain Adaptation [66.74638960925854]
  Partial domain adaptation (PDA) deals with a realistic and challenging problem when the source domain label space substitutes the target domain. We propose an Adaptively-Accumulated Knowledge Transfer framework (A$2$KT) to align the relevant categories across two domains.
  arXiv  Detail & Related papers  (2020-08-27T00:53:43Z)
• Dual Distribution Alignment Network for Generalizable Person Re-Identification [174.36157174951603]
  Domain generalization (DG) serves as a promising solution to handle person Re-Identification (Re-ID) We present a Dual Distribution Alignment Network (DDAN) which handles this challenge by selectively aligning distributions of multiple source domains. We evaluate our DDAN on a large-scale Domain Generalization Re-ID (DG Re-ID) benchmark.
  arXiv  Detail & Related papers  (2020-07-27T00:08:07Z)
• NestedVAE: Isolating Common Factors via Weak Supervision [45.366986365879505]
  We identify the connection between the task of bias reduction and that of isolating factors common between domains. To isolate the common factors we combine the theory of deep latent variable models with information bottleneck theory. Two outer VAEs with shared weights attempt to reconstruct the input and infer a latent space, whilst a nested VAE attempts to reconstruct the latent representation of one image, from the latent representation of its paired image.
  arXiv  Detail & Related papers  (2020-02-26T15:49:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.