Multi-Domain Causal Representation Learning via Weak Distributional Invariances

• URL: http://arxiv.org/abs/2310.02854v3
• Date: Mon, 11 Dec 2023 09:30:49 GMT
• Title: Multi-Domain Causal Representation Learning via Weak Distributional Invariances
• Authors: Kartik Ahuja, Amin Mansouri, Yixin Wang
• Abstract summary: Causal representation learning has emerged as the center of action in causal machine learning research. We show that autoencoders that incorporate such invariances can provably identify the stable set of latents from the rest across different settings.
• Score: 27.72497122405241
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Causal representation learning has emerged as the center of action in causal machine learning research. In particular, multi-domain datasets present a natural opportunity for showcasing the advantages of causal representation learning over standard unsupervised representation learning. While recent works have taken crucial steps towards learning causal representations, they often lack applicability to multi-domain datasets due to over-simplifying assumptions about the data; e.g. each domain comes from a different single-node perfect intervention. In this work, we relax these assumptions and capitalize on the following observation: there often exists a subset of latents whose certain distributional properties (e.g., support, variance) remain stable across domains; this property holds when, for example, each domain comes from a multi-node imperfect intervention. Leveraging this observation, we show that autoencoders that incorporate such invariances can provably identify the stable set of latents from the rest across different settings.

Related papers

• DIGIC: Domain Generalizable Imitation Learning by Causal Discovery [69.13526582209165]
  Causality has been combined with machine learning to produce robust representations for domain generalization. We make a different attempt by leveraging the demonstration data distribution to discover causal features for a domain generalizable policy. We design a novel framework, called DIGIC, to identify the causal features by finding the direct cause of the expert action from the demonstration data distribution.
  arXiv  Detail & Related papers  (2024-02-29T07:09:01Z)
• Continuous Invariance Learning [37.5006565403112]
  We show that existing invariance learning methods can fail for continuous domain problems. We propose Continuous Invariance Learning (CIL), which extracts invariant features across continuously indexed domains. CIL consistently outperforms strong baselines among all the tasks.
  arXiv  Detail & Related papers  (2023-10-09T02:18:45Z)
• DIVERSIFY: A General Framework for Time Series Out-of-distribution Detection and Generalization [58.704753031608625]
  Time series is one of the most challenging modalities in machine learning research. OOD detection and generalization on time series tend to suffer due to its non-stationary property. We propose DIVERSIFY, a framework for OOD detection and generalization on dynamic distributions of time series.
  arXiv  Detail & Related papers  (2023-08-04T12:27:11Z)
• Leveraging sparse and shared feature activations for disentangled representation learning [112.22699167017471]
  We propose to leverage knowledge extracted from a diversified set of supervised tasks to learn a common disentangled representation. We validate our approach on six real world distribution shift benchmarks, and different data modalities.
  arXiv  Detail & Related papers  (2023-04-17T01:33:24Z)
• Multi-view Adversarial Discriminator: Mine the Non-causal Factors for Object Detection in Unseen Domains [36.4342793435982]
  We present an idea to remove non-causal factors from common features by multi-view adversarial training on source domains. We propose a Multi-view Adversarial Discriminator (MAD) based domain generalization model. Our MAD obtains state-of-the-art performance on six benchmarks.
  arXiv  Detail & Related papers  (2023-04-06T09:20:28Z)
• Multi-Domain Long-Tailed Learning by Augmenting Disentangled Representations [80.76164484820818]
  There is an inescapable long-tailed class-imbalance issue in many real-world classification problems. We study this multi-domain long-tailed learning problem and aim to produce a model that generalizes well across all classes and domains. Built upon a proposed selective balanced sampling strategy, TALLY achieves this by mixing the semantic representation of one example with the domain-associated nuisances of another.
  arXiv  Detail & Related papers  (2022-10-25T21:54:26Z)
• Identifiable Latent Causal Content for Domain Adaptation under Latent Covariate Shift [82.14087963690561]
  Multi-source domain adaptation (MSDA) addresses the challenge of learning a label prediction function for an unlabeled target domain. We present an intricate causal generative model by introducing latent noises across domains, along with a latent content variable and a latent style variable. The proposed approach showcases exceptional performance and efficacy on both simulated and real-world datasets.
  arXiv  Detail & Related papers  (2022-08-30T11:25:15Z)
• Gated Domain Units for Multi-source Domain Generalization [14.643490853965385]
  Distribution shift (DS) occurs when a dataset at test time differs from the dataset at training time. We introduce a modular neural network layer consisting of Gated Domain Units (GDUs) that learn a representation for each latent elementary distribution. During inference, a weighted ensemble of learning machines can be created by comparing new observations with the representations of each elementary distribution.
  arXiv  Detail & Related papers  (2022-06-24T18:12:38Z)
• Instrumental Variable-Driven Domain Generalization with Unobserved Confounders [53.735614014067394]
  Domain generalization (DG) aims to learn from multiple source domains a model that can generalize well on unseen target domains. We propose an instrumental variable-driven DG method (IV-DG) by removing the bias of the unobserved confounders with two-stage learning. In the first stage, it learns the conditional distribution of the input features of one domain given input features of another domain. In the second stage, it estimates the relationship by predicting labels with the learned conditional distribution.
  arXiv  Detail & Related papers  (2021-10-04T13:32:57Z)
• Contrastive ACE: Domain Generalization Through Alignment of Causal Mechanisms [34.99779761100095]
  Domain generalization aims to learn knowledge invariant across different distributions. We consider the causal invariance of the average causal effect of the features to the labels.
  arXiv  Detail & Related papers  (2021-06-02T04:01:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.