Diffusion Based Causal Representation Learning

• URL: http://arxiv.org/abs/2311.05421v1
• Date: Thu, 9 Nov 2023 14:59:26 GMT
• Title: Diffusion Based Causal Representation Learning
• Authors: Amir Mohammad Karimi Mamaghan, Andrea Dittadi, Stefan Bauer, Karl Henrik Johansson, Francesco Quinzan
• Abstract summary: Causal reasoning can be considered a cornerstone of intelligent systems. Previous works on causal representation learning have mostly focused on Variational Auto-Encoders (VAE) We propose a new Diffusion-based Causal Representation Learning (DCRL) algorithm.
• Score: 27.58853186215212
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Causal reasoning can be considered a cornerstone of intelligent systems. Having access to an underlying causal graph comes with the promise of cause-effect estimation and the identification of efficient and safe interventions. However, learning causal representations remains a major challenge, due to the complexity of many real-world systems. Previous works on causal representation learning have mostly focused on Variational Auto-Encoders (VAE). These methods only provide representations from a point estimate, and they are unsuitable to handle high dimensions. To overcome these problems, we proposed a new Diffusion-based Causal Representation Learning (DCRL) algorithm. This algorithm uses diffusion-based representations for causal discovery. DCRL offers access to infinite dimensional latent codes, which encode different levels of information in the latent code. In a first proof of principle, we investigate the use of DCRL for causal representation learning. We further demonstrate experimentally that this approach performs comparably well in identifying the causal structure and causal variables.

Related papers

• Sanity Checking Causal Representation Learning on a Simple Real-World System [11.429106388558925]
  We evaluate methods for causal representation learning on a simple, real-world system where these methods are expected to work. We select methods representative of different approaches to CRL and find that they all fail to recover the underlying causal factors. Our efforts highlight the contrast between the theoretical promise of the state of the art and the challenges in its application.
  arXiv  Detail & Related papers  (2025-02-27T13:56:54Z)
• Optimal Causal Representations and the Causal Information Bottleneck [0.19799527196428243]
  The Information Bottleneck (IB) method is a widely used approach in representation learning. Traditional methods like IB are purely statistical and ignore underlying causal structures, making them ill-suited for causal tasks. We propose the Causal Information Bottleneck (CIB), a causal extension of the IB, which compresses a set of chosen variables while maintaining causal control over a target variable.
  arXiv  Detail & Related papers  (2024-10-01T09:21:29Z)
• Unifying Causal Representation Learning with the Invariance Principle [21.375611599649716]
  Causal representation learning aims at recovering latent causal variables from high-dimensional observations. We show that many causal representation learning approaches methodologically align their representations with inherent data symmetries. This paper clarifies the role of causal assumptions in the discovery of causal variables and shifts the focus to preserving data symmetries.
  arXiv  Detail & Related papers  (2024-09-04T14:51:36Z)
• Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environment [10.814585613336778]
  Causal representation learning aims to combine the core strengths of machine learning and causality. This thesis investigates what is possible for CRL without direct supervision, and thus contributes to its theoretical foundations.
  arXiv  Detail & Related papers  (2024-06-19T09:14:40Z)
• Simple Ingredients for Offline Reinforcement Learning [86.1988266277766]
  offline reinforcement learning algorithms have proven effective on datasets highly connected to the target downstream task. We show that existing methods struggle with diverse data: their performance considerably deteriorates as data collected for related but different tasks is simply added to the offline buffer. We show that scale, more than algorithmic considerations, is the key factor influencing performance.
  arXiv  Detail & Related papers  (2024-03-19T18:57:53Z)
• Analyzing Adversarial Inputs in Deep Reinforcement Learning [53.3760591018817]
  We present a comprehensive analysis of the characterization of adversarial inputs, through the lens of formal verification. We introduce a novel metric, the Adversarial Rate, to classify models based on their susceptibility to such perturbations. Our analysis empirically demonstrates how adversarial inputs can affect the safety of a given DRL system with respect to such perturbations.
  arXiv  Detail & Related papers  (2024-02-07T21:58:40Z)
• Multi-modal Causal Structure Learning and Root Cause Analysis [67.67578590390907]
  We propose Mulan, a unified multi-modal causal structure learning method for root cause localization. We leverage a log-tailored language model to facilitate log representation learning, converting log sequences into time-series data. We also introduce a novel key performance indicator-aware attention mechanism for assessing modality reliability and co-learning a final causal graph.
  arXiv  Detail & Related papers  (2024-02-04T05:50:38Z)
• A Causal Ordering Prior for Unsupervised Representation Learning [27.18951912984905]
  Causal representation learning argues that factors of variation in a dataset are, in fact, causally related. We propose a fully unsupervised representation learning method that considers a data generation process with a latent additive noise model.
  arXiv  Detail & Related papers  (2023-07-11T18:12:05Z)
• Principled Knowledge Extrapolation with GANs [92.62635018136476]
  We study counterfactual synthesis from a new perspective of knowledge extrapolation. We show that an adversarial game with a closed-form discriminator can be used to address the knowledge extrapolation problem. Our method enjoys both elegant theoretical guarantees and superior performance in many scenarios.
  arXiv  Detail & Related papers  (2022-05-21T08:39:42Z)
• Provable RL with Exogenous Distractors via Multistep Inverse Dynamics [85.52408288789164]
  Real-world applications of reinforcement learning (RL) require the agent to deal with high-dimensional observations such as those generated from a megapixel camera. Prior work has addressed such problems with representation learning, through which the agent can provably extract endogenous, latent state information from raw observations. However, such approaches can fail in the presence of temporally correlated noise in the observations.
  arXiv  Detail & Related papers  (2021-10-17T15:21:27Z)
• Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning [76.00395335702572]
  A central goal for AI and causality is the joint discovery of abstract representations and causal structure. Existing environments for studying causal induction are poorly suited for this objective because they have complicated task-specific causal graphs. In this work, our goal is to facilitate research in learning representations of high-level variables as well as causal structures among them.
  arXiv  Detail & Related papers  (2021-07-02T05:44:56Z)
• Weakly Supervised Disentangled Generative Causal Representation Learning [21.392372783459013]
  We show that previous methods with independent priors fail to disentangle causally related factors even under supervision. We propose a new disentangled learning method that enables causal controllable generation and causal representation learning.
  arXiv  Detail & Related papers  (2020-10-06T11:38:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.