Disentangling Dynamical Systems: Causal Representation Learning Meets Local Sparse Attention

• URL: http://arxiv.org/abs/2603.14483v1
• Date: Sun, 15 Mar 2026 17:03:34 GMT
• Title: Disentangling Dynamical Systems: Causal Representation Learning Meets Local Sparse Attention
• Abstract summary: We develop a novel identifiability theorem, leveraging causal representation learning, to uncover disentangled representations of system parameters.<n>We instantiate system parameter identification as a variational inference problem, leveraging a sparsity-regularised transformer to uncover state-dependent causal structures.
• Score: 14.165510655766944
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Parametric system identification methods estimate the parameters of explicitly defined physical systems from data. Yet, they remain constrained by the need to provide an explicit function space, typically through a predefined library of candidate functions chosen via available domain knowledge. In contrast, deep learning can demonstrably model systems of broad complexity with high fidelity, but black-box function approximation typically fails to yield explicit descriptive or disentangled representations revealing the structure of a system. We develop a novel identifiability theorem, leveraging causal representation learning, to uncover disentangled representations of system parameters without structural assumptions. We derive a graphical criterion specifying when system parameters can be uniquely disentangled from raw trajectory data, up to permutation and diffeomorphism. Crucially, our analysis demonstrates that global causal structures provide a lower bound on the disentanglement guarantees achievable when considering local state-dependent causal structures. We instantiate system parameter identification as a variational inference problem, leveraging a sparsity-regularised transformer to uncover state-dependent causal structures. We empirically validate our approach across four synthetic domains, demonstrating its ability to recover highly disentangled representations that baselines fail to recover. Corroborating our theoretical analysis, our results confirm that enforcing local causal structure is often necessary for full identifiability.

Related papers

• Neural operator discovery from heterogeneous trajectories [3.174604660846279]
  We introduce a factorized latent-conditioning formulation that jointly learns a neural operator and a low-dimensional latent representation.<n>Across diverse systems, the learned latent representation captures the intrinsic dimensionality of system variation.<n>Results establish an interpretable paradigm for operator learning in the absence of explicit factor supervision.
  arXiv  Detail & Related papers  (2026-07-25T19:20:25Z)
• Conditional Attribution for Root Cause Analysis in Time-Series Anomaly Detection [13.555771273974697]
  We propose a conditional attribution framework that explains anomalies relative to contextually similar normal system states.<n>Our method retrieves representative normal instances conditioned on the anomalous observation, enabling dependency-preserving and operationally meaningful explanations.<n> Experiments on the SWaT and benchmarks demonstrate that the proposed approach consistently improves root-cause identification accuracy, temporal localization, and robustness across multiple anomaly detection models.
  arXiv  Detail & Related papers  (2026-04-19T21:01:52Z)
• Anatomy of a Lie: A Multi-Stage Diagnostic Framework for Tracing Hallucinations in Vision-Language Models [62.932580559941414]
  Vision-Language Models (VLMs) frequently "hallucinate" - generate plausible yet factually incorrect statements.<n>We propose a new paradigm for diagnosing hallucinations, recasting them from static output errors into dynamic pathologies of a model's computational cognition.
  arXiv  Detail & Related papers  (2026-03-16T17:20:38Z)
• The Powers of Precision: Structure-Informed Detection in Complex Systems -- From Customer Churn to Seizure Onset [11.431576667955268]
  Emergent phenomena -- onset of epileptic seizures, sudden customer churn, or pandemic outbreaks -- often arise from hidden causal interactions in complex systems.<n>We propose a machine learning method for their early detection that addresses a core challenge: unveiling and harnessing a system's latent causal structure.
  arXiv  Detail & Related papers  (2026-01-29T02:11:44Z)
• Systemic Constraints of Undecidability [0.0]
  This paper presents a theory of systemic undecidability, reframing incomputability as a structural property of systems.<n>We prove a closure principle: any subsystem that participates functionally in the computation of an undecidable system inherits its undecidability.<n>Our framework disarms oracle mimicry and challenges the view that computational limits can be circumvented through architectural innovation.
  arXiv  Detail & Related papers  (2025-06-21T22:56:26Z)
• Detecting and Identifying Selection Structure in Sequential Data [53.24493902162797]
  We argue that the selective inclusion of data points based on latent objectives is common in practical situations, such as music sequences. We show that selection structure is identifiable without any parametric assumptions or interventional experiments. We also propose a provably correct algorithm to detect and identify selection structures as well as other types of dependencies.
  arXiv  Detail & Related papers  (2024-06-29T20:56:34Z)
• Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse Actions, Interventions and Sparse Temporal Dependencies [58.179981892921056]
  This work introduces a novel principle for disentanglement we call mechanism sparsity regularization. We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors. We show that the latent factors can be recovered by regularizing the learned causal graph to be sparse.
  arXiv  Detail & Related papers  (2024-01-10T02:38:21Z)
• Identifiability and Asymptotics in Learning Homogeneous Linear ODE Systems from Discrete Observations [114.17826109037048]
  Ordinary Differential Equations (ODEs) have recently gained a lot of attention in machine learning. theoretical aspects, e.g., identifiability and properties of statistical estimation are still obscure. This paper derives a sufficient condition for the identifiability of homogeneous linear ODE systems from a sequence of equally-spaced error-free observations sampled from a single trajectory.
  arXiv  Detail & Related papers  (2022-10-12T06:46:38Z)
• Discovering Latent Causal Variables via Mechanism Sparsity: A New Principle for Nonlinear ICA [81.4991350761909]
  Independent component analysis (ICA) refers to an ensemble of methods which formalize this goal and provide estimation procedure for practical application. We show that the latent variables can be recovered up to a permutation if one regularizes the latent mechanisms to be sparse.
  arXiv  Detail & Related papers  (2021-07-21T14:22:14Z)
• Structural Causal Models Are (Solvable by) Credal Networks [70.45873402967297]
  Causal inferences can be obtained by standard algorithms for the updating of credal nets. This contribution should be regarded as a systematic approach to represent structural causal models by credal networks. Experiments show that approximate algorithms for credal networks can immediately be used to do causal inference in real-size problems.
  arXiv  Detail & Related papers  (2020-08-02T11:19:36Z)
• On Linear Identifiability of Learned Representations [26.311880922890843]
  We study identifiability in the context of representation learning. We show that a large family of discriminative models are identifiable in function space, up to a linear indeterminacy. We derive sufficient conditions for linear identifiability and provide empirical support for the result on both simulated and real-world data.
  arXiv  Detail & Related papers  (2020-07-01T23:33:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.