Partial Disentanglement via Mechanism Sparsity

• URL: http://arxiv.org/abs/2207.07732v1
• Date: Fri, 15 Jul 2022 20:06:12 GMT
• Title: Partial Disentanglement via Mechanism Sparsity
• Authors: S\'ebastien Lachapelle and Simon Lacoste-Julien
• Abstract summary: Disentanglement via mechanism sparsity was introduced as a principled approach to extract latent factors without supervision. We introduce a generalization of this theory which applies to any ground-truth graph. We show how disentangled the learned representation is expected to be, via a new equivalence relation over models we call consistency.
• Score: 25.791043728989937
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Disentanglement via mechanism sparsity was introduced recently as a principled approach to extract latent factors without supervision when the causal graph relating them in time is sparse, and/or when actions are observed and affect them sparsely. However, this theory applies only to ground-truth graphs satisfying a specific criterion. In this work, we introduce a generalization of this theory which applies to any ground-truth graph and specifies qualitatively how disentangled the learned representation is expected to be, via a new equivalence relation over models we call consistency. This equivalence captures which factors are expected to remain entangled and which are not based on the specific form of the ground-truth graph. We call this weaker form of identifiability partial disentanglement. The graphical criterion that allows complete disentanglement, proposed in an earlier work, can be derived as a special case of our theory. Finally, we enforce graph sparsity with constrained optimization and illustrate our theory and algorithm in simulations.

Related papers

• Identifiable Latent Neural Causal Models [82.14087963690561]
  Causal representation learning seeks to uncover latent, high-level causal representations from low-level observed data. We determine the types of distribution shifts that do contribute to the identifiability of causal representations. We translate our findings into a practical algorithm, allowing for the acquisition of reliable latent causal representations.
  arXiv  Detail & Related papers  (2024-03-23T04:13:55Z)
• Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse Actions, Interventions and Sparse Temporal Dependencies [58.179981892921056]
  This work introduces a novel principle for disentanglement we call mechanism sparsity regularization. We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors. We show that the latent factors can be recovered by regularizing the learned causal graph to be sparse.
  arXiv  Detail & Related papers  (2024-01-10T02:38:21Z)
• The Generative Programs Framework [0.0]
  We argue that any quantitative physical theory can be represented in the form of a generative program. We suggest that these graphs can be interpreted as encoding relations of ontological priority,' and that ontological priority is a suitable generalisation of causation.
  arXiv  Detail & Related papers  (2023-07-21T00:57:05Z)
• Axiomatization of Interventional Probability Distributions [4.02487511510606]
  Causal intervention is axiomatized under the rules of do-calculus. We show that under our axiomatizations, the intervened distributions are Markovian to the defined intervened causal graphs. We also show that a large class of natural structural causal models satisfy the theory presented here.
  arXiv  Detail & Related papers  (2023-05-08T06:07:42Z)
• Understanding convolution on graphs via energies [23.18124653469668]
  Graph Networks (GNNs) typically operate by message-passing, where the state of a node is updated based on the information received from its neighbours. Most message-passing models act as graph convolutions, where features are mixed by a shared, linear transformation before being propagated over the edges. On node-classification tasks, graph convolutions have been shown to suffer from two limitations: poor performance on heterophilic graphs, and over-smoothing.
  arXiv  Detail & Related papers  (2022-06-22T11:45:36Z)
• Invariance Principle Meets Out-of-Distribution Generalization on Graphs [66.04137805277632]
  Complex nature of graphs thwarts the adoption of the invariance principle for OOD generalization. domain or environment partitions, which are often required by OOD methods, can be expensive to obtain for graphs. We propose a novel framework to explicitly model this process using a contrastive strategy.
  arXiv  Detail & Related papers  (2022-02-11T04:38:39Z)
• Typing assumptions improve identification in causal discovery [123.06886784834471]
  Causal discovery from observational data is a challenging task to which an exact solution cannot always be identified. We propose a new set of assumptions that constrain possible causal relationships based on the nature of the variables.
  arXiv  Detail & Related papers  (2021-07-22T14:23:08Z)
• Discovering Latent Causal Variables via Mechanism Sparsity: A New Principle for Nonlinear ICA [81.4991350761909]
  Independent component analysis (ICA) refers to an ensemble of methods which formalize this goal and provide estimation procedure for practical application. We show that the latent variables can be recovered up to a permutation if one regularizes the latent mechanisms to be sparse.
  arXiv  Detail & Related papers  (2021-07-21T14:22:14Z)
• Conditions and Assumptions for Constraint-based Causal Structure Learning [2.741266294612776]
  The paper formalizes constraint-based structure learning of the "true" causal graph from observed data. We provide the theory for the general class of models under the assumption that the distribution is Markovian to the true causal graph.
  arXiv  Detail & Related papers  (2021-03-24T23:08:00Z)
• Causal Expectation-Maximisation [70.45873402967297]
  We show that causal inference is NP-hard even in models characterised by polytree-shaped graphs. We introduce the causal EM algorithm to reconstruct the uncertainty about the latent variables from data about categorical manifest variables. We argue that there appears to be an unnoticed limitation to the trending idea that counterfactual bounds can often be computed without knowledge of the structural equations.
  arXiv  Detail & Related papers  (2020-11-04T10:25:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.