Identifiability of latent-variable and structural-equation models: from linear to nonlinear

• URL: http://arxiv.org/abs/2302.02672v2
• Date: Wed, 3 May 2023 08:35:48 GMT
• Title: Identifiability of latent-variable and structural-equation models: from linear to nonlinear
• Authors: Aapo Hyv\"arinen, Ilyes Khemakhem, Ricardo Monti
• Abstract summary: In factor analysis, non-Gaussianity of the (latent) variables has been shown to provide identifiability. More recently, we have shown how even general non nonlinear versions of such models can be estimated.
• Score: 2.159277717031637
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: An old problem in multivariate statistics is that linear Gaussian models are often unidentifiable, i.e. some parameters cannot be uniquely estimated. In factor (component) analysis, an orthogonal rotation of the factors is unidentifiable, while in linear regression, the direction of effect cannot be identified. For such linear models, non-Gaussianity of the (latent) variables has been shown to provide identifiability. In the case of factor analysis, this leads to independent component analysis, while in the case of the direction of effect, non-Gaussian versions of structural equation modelling solve the problem. More recently, we have shown how even general nonparametric nonlinear versions of such models can be estimated. Non-Gaussianity is not enough in this case, but assuming we have time series, or that the distributions are suitably modulated by some observed auxiliary variables, the models are identifiable. This paper reviews the identifiability theory for the linear and nonlinear cases, considering both factor analytic models and structural equation models.

Related papers

• Parameter identification in linear non-Gaussian causal models under general confounding [8.273471398838533]
  We study identification of the linear coefficients when such models contain latent variables. Our main result is a graphical criterion that is necessary and sufficient for deciding generic identifiability of direct causal effects. We report on estimations based on the identification result, explore a generalization to models with feedback loops, and provide new results on the identifiability of the causal graph.
  arXiv  Detail & Related papers  (2024-05-31T14:39:14Z)
• Scaling and renormalization in high-dimensional regression [72.59731158970894]
  This paper presents a succinct derivation of the training and generalization performance of a variety of high-dimensional ridge regression models. We provide an introduction and review of recent results on these topics, aimed at readers with backgrounds in physics and deep learning.
  arXiv  Detail & Related papers  (2024-05-01T15:59:00Z)
• Learning Graphical Factor Models with Riemannian Optimization [70.13748170371889]
  This paper proposes a flexible algorithmic framework for graph learning under low-rank structural constraints. The problem is expressed as penalized maximum likelihood estimation of an elliptical distribution. We leverage geometries of positive definite matrices and positive semi-definite matrices of fixed rank that are well suited to elliptical models.
  arXiv  Detail & Related papers  (2022-10-21T13:19:45Z)
• Function Classes for Identifiable Nonlinear Independent Component Analysis [10.828616610785524]
  Unsupervised learning of latent variable models (LVMs) is widely used to represent data in machine learning. Recent work suggests that constraining the function class of such models may promote identifiability. We prove that a subclass of these transformations, conformal maps, is identifiable and provide novel theoretical results.
  arXiv  Detail & Related papers  (2022-08-12T17:58:31Z)
• On the Identifiability of Nonlinear ICA: Sparsity and Beyond [20.644375143901488]
  How to make the nonlinear ICA model identifiable up to certain trivial indeterminacies is a long-standing problem in unsupervised learning. Recent breakthroughs reformulate the standard independence assumption of sources as conditional independence given some auxiliary variables. We show that under specific instantiations of such constraints, the independent latent sources can be identified from their nonlinear mixtures up to a permutation.
  arXiv  Detail & Related papers  (2022-06-15T18:24:22Z)
• Estimation of Bivariate Structural Causal Models by Variational Gaussian Process Regression Under Likelihoods Parametrised by Normalising Flows [74.85071867225533]
  Causal mechanisms can be described by structural causal models. One major drawback of state-of-the-art artificial intelligence is its lack of explainability.
  arXiv  Detail & Related papers  (2021-09-06T14:52:58Z)
• Hessian Eigenspectra of More Realistic Nonlinear Models [73.31363313577941]
  We make a emphprecise characterization of the Hessian eigenspectra for a broad family of nonlinear models. Our analysis takes a step forward to identify the origin of many striking features observed in more complex machine learning models.
  arXiv  Detail & Related papers  (2021-03-02T06:59:52Z)
• Sparse Quantized Spectral Clustering [85.77233010209368]
  We exploit tools from random matrix theory to make precise statements about how the eigenspectrum of a matrix changes under such nonlinear transformations. We show that very little change occurs in the informative eigenstructure even under drastic sparsification/quantization.
  arXiv  Detail & Related papers  (2020-10-03T15:58:07Z)
• Non-parametric Models for Non-negative Functions [48.7576911714538]
  We provide the first model for non-negative functions from the same good linear models. We prove that it admits a representer theorem and provide an efficient dual formulation for convex problems.
  arXiv  Detail & Related papers  (2020-07-08T07:17:28Z)
• Latent variable modeling with random features [7.856578780790166]
  We develop a family of nonlinear dimension reduction models that are easily to non-Gaussian data likelihoods. Our generalized RFLVMs produce results comparable with other state-of-the-art dimension reduction methods.
  arXiv  Detail & Related papers  (2020-06-19T14:12:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.