Linear Causal Representation Learning by Topological Ordering, Pruning, and Disentanglement

• URL: http://arxiv.org/abs/2509.22553v1
• Date: Fri, 26 Sep 2025 16:35:42 GMT
• Title: Linear Causal Representation Learning by Topological Ordering, Pruning, and Disentanglement
• Authors: Hao Chen, Lin Liu, Yu Guang Wang,
• Abstract summary: Causal representation learning (CRL) has garnered increasing interests from the causal inference and artificial intelligence community.<n>We propose a novel linear CRL algorithm that operates under weaker assumptions about environment heterogeneity and data-generating distributions.<n>We validate our new algorithm via synthetic experiments and an interpretability analysis of large language models.
• Score: 12.380741069149956
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Causal representation learning (CRL) has garnered increasing interests from the causal inference and artificial intelligence community, due to its capability of disentangling potentially complex data-generating mechanism into causally interpretable latent features, by leveraging the heterogeneity of modern datasets. In this paper, we further contribute to the CRL literature, by focusing on the stylized linear structural causal model over the latent features and assuming a linear mixing function that maps latent features to the observed data or measurements. Existing linear CRL methods often rely on stringent assumptions, such as accessibility to single-node interventional data or restrictive distributional constraints on latent features and exogenous measurement noise. However, these prerequisites can be challenging to satisfy in certain scenarios. In this work, we propose a novel linear CRL algorithm that, unlike most existing linear CRL methods, operates under weaker assumptions about environment heterogeneity and data-generating distributions while still recovering latent causal features up to an equivalence class. We further validate our new algorithm via synthetic experiments and an interpretability analysis of large language models (LLMs), demonstrating both its superiority over competing methods in finite samples and its potential in integrating causality into AI.

Related papers

• FlexCausal: Flexible Causal Disentanglement via Structural Flow Priors and Manifold-Aware Interventions [1.7114074082429929]
  Causal Disentangled Representation Learning aims to learn and disentangle low dimensional representations from observations.<n>We propose FlexCausal, a novel CDRL framework based on a block-diagonal covariance VAE.<n>Our framework ensures a precise structural correspondence between the learned latent subspaces and the ground-truth causal relations.
  arXiv  Detail & Related papers  (2026-01-29T11:30:53Z)
• Latent Causal Diffusions for Single-Cell Perturbation Modeling [83.47931153555321]
  We present a generative model that frames single-cell gene expression as a stationary diffusion process observed under measurement noise.<n> LCD outperforms established approaches in predicting the distributional shifts of unseen perturbation combinations in single-cell RNA-sequencing screens.<n>We develop an approach we call causal linearization via perturbation responses (CLIPR), which yields an approximation of the direct causal effects between all genes.
  arXiv  Detail & Related papers  (2026-01-20T16:15:38Z)
• SIGMA: Scalable Spectral Insights for LLM Collapse [51.863164847253366]
  We introduce SIGMA (Spectral Inequalities for Gram Matrix Analysis), a unified framework for model collapse.<n>By utilizing benchmarks that deriving and deterministic bounds on the matrix's spectrum, SIGMA provides a mathematically grounded metric to track the contraction of the representation space.<n>We demonstrate that SIGMA effectively captures the transition towards states, offering both theoretical insights into the mechanics of collapse.
  arXiv  Detail & Related papers  (2026-01-06T19:47:11Z)
• Learning Causality for Longitudinal Data [1.2691047660244335]
  This thesis develops methods for causal inference and causal representation learning in high-dimensional, time-varying data.<n>The first contribution introduces the Causal Dynamic Variational Autoencoder (CDVAE), a model for estimating Individual Treatment Effects (ITEs)<n>The second contribution proposes an efficient framework for long-term counterfactual regression based on RNNs enhanced with Contrastive Predictive Coding ( CPC) and InfoMax.<n>The third contribution advances CRL by addressing how latent causes manifest in observed variables.
  arXiv  Detail & Related papers  (2025-12-04T16:51:49Z)
• On the Identifiability of Causal Abstractions [15.785002371773139]
  Causal representation learning enhances machine learning models' robustness and generalizability.<n>We focus on a family of CRL methods that uses contrastive data pairs in the observable space to identify the latent causal model.<n>We introduce a theoretical framework that calculates the degree to which we can identify a causal model, given a set of possible interventions.
  arXiv  Detail & Related papers  (2025-03-13T19:34:05Z)
• Induced Covariance for Causal Discovery in Linear Sparse Structures [55.2480439325792]
  Causal models seek to unravel the cause-effect relationships among variables from observed data. This paper introduces a novel causal discovery algorithm designed for settings in which variables exhibit linearly sparse relationships.
  arXiv  Detail & Related papers  (2024-10-02T04:01:38Z)
• Hybrid Top-Down Global Causal Discovery with Local Search for Linear and Nonlinear Additive Noise Models [2.0738462952016232]
  Methods based on functional causal models can identify a unique graph, but suffer from the curse of dimensionality or impose strong parametric assumptions.<n>We propose a novel hybrid approach for global causal discovery in observational data that leverages local causal substructures.<n>We provide theoretical guarantees for correctness and worst-case time complexities, with empirical validation on synthetic data.
  arXiv  Detail & Related papers  (2024-05-23T12:28:16Z)
• Neural Network Approximation for Pessimistic Offline Reinforcement Learning [17.756108291816908]
  We present a non-asymptotic estimation error of pessimistic offline RL using general neural network approximation. Our result shows that the estimation error consists of two parts: the first converges to zero at a desired rate on the sample size with partially controllable concentrability, and the second becomes negligible if the residual constraint is tight.
  arXiv  Detail & Related papers  (2023-12-19T05:17:27Z)
• GEC: A Unified Framework for Interactive Decision Making in MDP, POMDP, and Beyond [101.5329678997916]
  We study sample efficient reinforcement learning (RL) under the general framework of interactive decision making. We propose a novel complexity measure, generalized eluder coefficient (GEC), which characterizes the fundamental tradeoff between exploration and exploitation. We show that RL problems with low GEC form a remarkably rich class, which subsumes low Bellman eluder dimension problems, bilinear class, low witness rank problems, PO-bilinear class, and generalized regular PSR.
  arXiv  Detail & Related papers  (2022-11-03T16:42:40Z)
• Learning Neural Causal Models with Active Interventions [83.44636110899742]
  We introduce an active intervention-targeting mechanism which enables a quick identification of the underlying causal structure of the data-generating process. Our method significantly reduces the required number of interactions compared with random intervention targeting. We demonstrate superior performance on multiple benchmarks from simulated to real-world data.
  arXiv  Detail & Related papers  (2021-09-06T13:10:37Z)
• Discovering Latent Causal Variables via Mechanism Sparsity: A New Principle for Nonlinear ICA [81.4991350761909]
  Independent component analysis (ICA) refers to an ensemble of methods which formalize this goal and provide estimation procedure for practical application. We show that the latent variables can be recovered up to a permutation if one regularizes the latent mechanisms to be sparse.
  arXiv  Detail & Related papers  (2021-07-21T14:22:14Z)
• Multiplicative noise and heavy tails in stochastic optimization [62.993432503309485]
  empirical optimization is central to modern machine learning, but its role in its success is still unclear. We show that it commonly arises in parameters of discrete multiplicative noise due to variance. A detailed analysis is conducted in which we describe on key factors, including recent step size, and data, all exhibit similar results on state-of-the-art neural network models.
  arXiv  Detail & Related papers  (2020-06-11T09:58:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.