Identifiability of a statistical model with two latent vectors: Importance of the dimensionality relation and application to graph embedding

• URL: http://arxiv.org/abs/2405.19760v1
• Date: Thu, 30 May 2024 07:11:20 GMT
• Title: Identifiability of a statistical model with two latent vectors: Importance of the dimensionality relation and application to graph embedding
• Authors: Hiroaki Sasaki,
• Abstract summary: Identifiability of statistical models is a key notion in unsupervised representation learning. This paper proposes a statistical model of two latent vectors with single auxiliary data generalizing nonlinear ICA. Surprisingly, we prove that the indeterminacies of the proposed model has the same as emphlinear ICA under certain conditions.
• Score: 2.6651200086513107
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Identifiability of statistical models is a key notion in unsupervised representation learning. Recent work of nonlinear independent component analysis (ICA) employs auxiliary data and has established identifiable conditions. This paper proposes a statistical model of two latent vectors with single auxiliary data generalizing nonlinear ICA, and establishes various identifiability conditions. Unlike previous work, the two latent vectors in the proposed model can have arbitrary dimensions, and this property enables us to reveal an insightful dimensionality relation among two latent vectors and auxiliary data in identifiability conditions. Furthermore, surprisingly, we prove that the indeterminacies of the proposed model has the same as \emph{linear} ICA under certain conditions: The elements in the latent vector can be recovered up to their permutation and scales. Next, we apply the identifiability theory to a statistical model for graph data. As a result, one of the identifiability conditions includes an appealing implication: Identifiability of the statistical model could depend on the maximum value of link weights in graph data. Then, we propose a practical method for identifiable graph embedding. Finally, we numerically demonstrate that the proposed method well-recovers the latent vectors and model identifiability clearly depends on the maximum value of link weights, which supports the implication of our theoretical results

Related papers

• Cyclic Directed Probabilistic Graphical Model: A Proposal Based on Structured Outcomes [0.0]
  We describe a probabilistic graphical model - probabilistic relation network - that allows the direct capture of directional cyclic dependencies. This model does not violate the probability axioms, and it supports learning from observed data. Notably, it supports probabilistic inference, making it a prospective tool in data analysis and in expert and design-making applications.
  arXiv  Detail & Related papers  (2023-10-25T10:19:03Z)
• Goodness-of-Fit of Attributed Probabilistic Graph Generative Models [11.58149447373971]
  We define goodness of fit in terms of the mean square contingency coefficient for random binary networks. We apply these criteria to verify the representation capability of a probabilistic generative model for various popular types of graph models.
  arXiv  Detail & Related papers  (2023-07-28T18:48:09Z)
• Sufficient Identification Conditions and Semiparametric Estimation under Missing Not at Random Mechanisms [4.211128681972148]
  Conducting valid statistical analyses is challenging in the presence of missing-not-at-random (MNAR) data. We consider a MNAR model that generalizes several prior popular MNAR models in two ways. We propose methods for testing the independence restrictions encoded in such models using odds ratio as our parameter of interest.
  arXiv  Detail & Related papers  (2023-06-10T13:46:16Z)
• ChiroDiff: Modelling chirographic data with Diffusion Models [132.5223191478268]
  We introduce a powerful model-class namely "Denoising Diffusion Probabilistic Models" or DDPMs for chirographic data. Our model named "ChiroDiff", being non-autoregressive, learns to capture holistic concepts and therefore remains resilient to higher temporal sampling rate.
  arXiv  Detail & Related papers  (2023-04-07T15:17:48Z)
• MAUVE Scores for Generative Models: Theory and Practice [95.86006777961182]
  We present MAUVE, a family of comparison measures between pairs of distributions such as those encountered in the generative modeling of text or images. We find that MAUVE can quantify the gaps between the distributions of human-written text and those of modern neural language models. We demonstrate in the vision domain that MAUVE can identify known properties of generated images on par with or better than existing metrics.
  arXiv  Detail & Related papers  (2022-12-30T07:37:40Z)
• Staged trees and asymmetry-labeled DAGs [2.66269503676104]
  We introduce a minimal Bayesian network representation of the staged tree, which can be used to read conditional independences in an intuitive way. We also define a new labeled graph, termed asymmetry-labeled directed acyclic graph, whose edges are labeled to denote the type of dependence existing between any two random variables.
  arXiv  Detail & Related papers  (2021-08-04T12:20:47Z)
• Typing assumptions improve identification in causal discovery [123.06886784834471]
  Causal discovery from observational data is a challenging task to which an exact solution cannot always be identified. We propose a new set of assumptions that constrain possible causal relationships based on the nature of the variables.
  arXiv  Detail & Related papers  (2021-07-22T14:23:08Z)
• PSD Representations for Effective Probability Models [117.35298398434628]
  We show that a recently proposed class of positive semi-definite (PSD) models for non-negative functions is particularly suited to this end. We characterize both approximation and generalization capabilities of PSD models, showing that they enjoy strong theoretical guarantees. Our results open the way to applications of PSD models to density estimation, decision theory and inference.
  arXiv  Detail & Related papers  (2021-06-30T15:13:39Z)
• On Linear Identifiability of Learned Representations [26.311880922890843]
  We study identifiability in the context of representation learning. We show that a large family of discriminative models are identifiable in function space, up to a linear indeterminacy. We derive sufficient conditions for linear identifiability and provide empirical support for the result on both simulated and real-world data.
  arXiv  Detail & Related papers  (2020-07-01T23:33:37Z)
• Asymptotic Analysis of an Ensemble of Randomly Projected Linear Discriminants [94.46276668068327]
  In [1], an ensemble of randomly projected linear discriminants is used to classify datasets. We develop a consistent estimator of the misclassification probability as an alternative to the computationally-costly cross-validation estimator. We also demonstrate the use of our estimator for tuning the projection dimension on both real and synthetic data.
  arXiv  Detail & Related papers  (2020-04-17T12:47:04Z)
• Block-Approximated Exponential Random Graphs [77.4792558024487]
  An important challenge in the field of exponential random graphs (ERGs) is the fitting of non-trivial ERGs on large graphs. We propose an approximative framework to such non-trivial ERGs that result in dyadic independence (i.e., edge independent) distributions. Our methods are scalable to sparse graphs consisting of millions of nodes.
  arXiv  Detail & Related papers  (2020-02-14T11:42:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.