Disentanglement Beyond Generative Models with Riemannian ICA

• URL: http://arxiv.org/abs/2605.22531v1
• Date: Thu, 21 May 2026 14:22:25 GMT
• Title: Disentanglement Beyond Generative Models with Riemannian ICA
• Abstract summary: We provide a theoretical basis for studying local disentanglement without assuming a global generative model.<n>Our main contribution is the disentanglement tensor, which encodes a second-order notion of disentanglement.
• Score: 1.3198689566654105
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: There is a gap between the theoretical foundations of disentanglement and the practice of modern representation learning. Existing theoretical frameworks, particularly Independent Component Analysis (ICA) and its nonlinear variants, assume a generative model with statistically independent latent variables underlying the data so that disentanglement amounts to identifying the latents that could have generated the data. This generative framework is interpretable and theoretically justified, but its strong assumptions make it difficult to apply to modern representation learning. Modern pretrained encoders often learn features that exhibit disentangled properties without making generative assumptions, yet there is no general theory for interpreting these features as independent factors of variation. We take a step toward such a theory by introducing Riemannian ICA (RICA), which replaces ICA's global generative model with local geometric structure. RICA is founded on the observation that in ICA, the factors of variation underlying a data point can be understood through radial curves emanating from the point that map to axis-aligned lines in the latent space. We formalize this perspective using Riemannian geometry and introduce our theory in a way that is consistent with the existing generative approach. Our main contribution is the disentanglement tensor, which encodes a second-order notion of disentanglement that we call pointwise disentanglement. This tensor depends on the Hessian of the data log likelihood as well as the Ricci curvature induced by the model. In a controlled source recovery setting with known ground-truth sources, RICA recovers sources across several manifolds, while the success of ICA baselines depends on the coordinates used to represent the observations. Our work provides a theoretical basis for studying local disentanglement without assuming a global generative model.

Related papers

• On Higher-Order Geometric Refinements of Classical Covariance Asymptotics: An Approach via Intrinsic and Extrinsic Information Geometry [0.0]
  In curved models, including mixtures, curved exponential families, latent-variable models, and manifold-root parameter spaces, finite-sample behavior can deviate systematically from predictions.<n>We develop a coordinate-invariant, curvature-aware refinement by viewing a regular parametric family as a ((,g)) with Fisher-constrained-Rao metric.<n>We describe the resolved metric, the role of the real log canonical threshold in learning rates and posterior mean-squared error, and a curvature-based covariance expansion on the resolved space that recovers the regular theory as a special case.
  arXiv  Detail & Related papers  (2026-04-14T13:40:13Z)
• Distributional Equivalence in Linear Non-Gaussian Latent-Variable Cyclic Causal Models: Characterization and Learning [13.891913455492697]
  We argue that a core obstacle to a general, structural-assumption-free approach is the lack of an equivalence characterization.<n>Key to our approach is a new tool, edge rank constraints, which fills a missing piece in the toolbox for latent-variable causal discovery.
  arXiv  Detail & Related papers  (2026-03-05T03:57:14Z)
• Generalization Below the Edge of Stability: The Role of Data Geometry [60.147710896851045]
  We show how data geometry controls generalization in ReLU networks trained below the edge of stability.<n>For data distributions supported on a mixture of low-dimensional balls, we derive generalization bounds that provably adapt to the intrinsic dimension.<n>Our results consolidate disparate empirical findings that have appeared in the literature.
  arXiv  Detail & Related papers  (2025-10-20T21:40:36Z)
• Learning Latent Graph Geometry via Fixed-Point Schrödinger-Type Activation: A Theoretical Study [1.1745324895296467]
  We develop a unified theoretical framework for neural architectures with internal representations evolving as stationary states of dissipative Schr"odinger-type dynamics on learned latent graphs.<n>We prove existence, uniqueness, and smooth dependence of equilibria, and show that the dynamics are equivalent under the Bloch map to norm-preserving Landau--Lifshitz flows.<n>The resulting model class provides a compact, geometrically interpretable, and analytically tractable foundation for learning latent graph geometry via fixed-point Schr"odinger-type activations.
  arXiv  Detail & Related papers  (2025-07-27T00:35:15Z)
• Wrapped Gaussian on the manifold of Symmetric Positive Definite Matrices [4.678796432640703]
  Circular and non-flat data distributions are prevalent across diverse domains of data science.<n>A principled approach to accounting for the underlying geometry of such data is pivotal.<n>This work lays the groundwork for extending classical machine learning and statistical methods to more complex and structured data.
  arXiv  Detail & Related papers  (2025-02-03T16:46:46Z)
• Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse Actions, Interventions and Sparse Temporal Dependencies [58.179981892921056]
  This work introduces a novel principle for disentanglement we call mechanism sparsity regularization. We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors. We show that the latent factors can be recovered by regularizing the learned causal graph to be sparse.
  arXiv  Detail & Related papers  (2024-01-10T02:38:21Z)
• On the Joint Interaction of Models, Data, and Features [82.60073661644435]
  We introduce a new tool, the interaction tensor, for empirically analyzing the interaction between data and model through features. Based on these observations, we propose a conceptual framework for feature learning. Under this framework, the expected accuracy for a single hypothesis and agreement for a pair of hypotheses can both be derived in closed-form.
  arXiv  Detail & Related papers  (2023-06-07T21:35:26Z)
• Disentanglement via Latent Quantization [60.37109712033694]
  In this work, we construct an inductive bias towards encoding to and decoding from an organized latent space. We demonstrate the broad applicability of this approach by adding it to both basic data-re (vanilla autoencoder) and latent-reconstructing (InfoGAN) generative models.
  arXiv  Detail & Related papers  (2023-05-28T06:30:29Z)
• Discovering Latent Causal Variables via Mechanism Sparsity: A New Principle for Nonlinear ICA [81.4991350761909]
  Independent component analysis (ICA) refers to an ensemble of methods which formalize this goal and provide estimation procedure for practical application. We show that the latent variables can be recovered up to a permutation if one regularizes the latent mechanisms to be sparse.
  arXiv  Detail & Related papers  (2021-07-21T14:22:14Z)
• GELATO: Geometrically Enriched Latent Model for Offline Reinforcement Learning [54.291331971813364]
  offline reinforcement learning approaches can be divided into proximal and uncertainty-aware methods. In this work, we demonstrate the benefit of combining the two in a latent variational model. Our proposed metrics measure both the quality of out of distribution samples as well as the discrepancy of examples in the data.
  arXiv  Detail & Related papers  (2021-02-22T19:42:40Z)
• Identifying the latent space geometry of network models through analysis of curvature [7.644165047073435]
  We present a method to consistently estimate the manifold type, dimension, and curvature from an empirically relevant class of latent spaces. Our core insight comes by representing the graph as a noisy distance matrix based on the ties between cliques.
  arXiv  Detail & Related papers  (2020-12-19T00:35:29Z)
• Generative Model without Prior Distribution Matching [26.91643368299913]
  Variational Autoencoder (VAE) and its variations are classic generative models by learning a low-dimensional latent representation to satisfy some prior distribution. We propose to let the prior match the embedding distribution rather than imposing the latent variables to fit the prior.
  arXiv  Detail & Related papers  (2020-09-23T09:33:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.