Identifying metric structures of deep latent variable models

• URL: http://arxiv.org/abs/2502.13757v2
• Date: Thu, 20 Feb 2025 09:52:58 GMT
• Title: Identifying metric structures of deep latent variable models
• Authors: Stas Syrota, Yevgen Zainchkovskyy, Johnny Xi, Benjamin Bloem-Reddy, Søren Hauberg,
• Abstract summary: Deep latent variable models learn condensed representations of data that, hopefully, reflect the inner workings of the studied phenomena. Current solutions limit the lack of identifiability through additional constraints on the latent variable model. We change the goal: instead of identifying the latent variables, we identify relationships between them.
• Score: 11.406154623608927
• License:
• Abstract: Deep latent variable models learn condensed representations of data that, hopefully, reflect the inner workings of the studied phenomena. Unfortunately, these latent representations are not statistically identifiable, meaning they cannot be uniquely determined. Domain experts, therefore, need to tread carefully when interpreting these. Current solutions limit the lack of identifiability through additional constraints on the latent variable model, e.g. by requiring labeled training data, or by restricting the expressivity of the model. We change the goal: instead of identifying the latent variables, we identify relationships between them such as meaningful distances, angles, and volumes. We prove this is feasible under very mild model conditions and without additional labeled data. We empirically demonstrate that our theory results in more reliable latent distances, offering a principled path forward in extracting trustworthy conclusions from deep latent variable models.

Related papers

• Learning Discrete Latent Variable Structures with Tensor Rank Conditions [30.292492090200984]
  Unobserved discrete data are ubiquitous in many scientific disciplines, and how to learn the causal structure of these latent variables is crucial for uncovering data patterns. Most studies focus on the linear latent variable model or impose strict constraints on latent structures, which fail to address cases in discrete data involving non-linear relationships or complex latent structures. We explore a tensor rank condition on contingency tables for an observed variable set $mathbfX_p$, showing that the rank is determined by the minimum support of a specific conditional set. One can locate the latent variable through probing the rank on different observed variables
  arXiv  Detail & Related papers  (2024-06-11T07:25:17Z)
• Nonparametric Identifiability of Causal Representations from Unknown Interventions [63.1354734978244]
  We study causal representation learning, the task of inferring latent causal variables and their causal relations from mixtures of the variables. Our goal is to identify both the ground truth latents and their causal graph up to a set of ambiguities which we show to be irresolvable from interventional data.
  arXiv  Detail & Related papers  (2023-06-01T10:51:58Z)
• Posterior Collapse and Latent Variable Non-identifiability [54.842098835445]
  We propose a class of latent-identifiable variational autoencoders, deep generative models which enforce identifiability without sacrificing flexibility. Across synthetic and real datasets, latent-identifiable variational autoencoders outperform existing methods in mitigating posterior collapse and providing meaningful representations of the data.
  arXiv  Detail & Related papers  (2023-01-02T06:16:56Z)
• Weakly Supervised Representation Learning with Sparse Perturbations [82.39171485023276]
  We show that if one has weak supervision from observations generated by sparse perturbations of the latent variables, identification is achievable under unknown continuous latent distributions. We propose a natural estimation procedure based on this theory and illustrate it on low-dimensional synthetic and image-based experiments.
  arXiv  Detail & Related papers  (2022-06-02T15:30:07Z)
• Indeterminacy in Latent Variable Models: Characterization and Strong Identifiability [3.959606869996233]
  We construct a theoretical framework for analyzing the indeterminacies of latent variable models. We then investigate how we might specify strongly identifiable latent variable models.
  arXiv  Detail & Related papers  (2022-06-02T00:01:27Z)
• Learning Conditional Invariance through Cycle Consistency [60.85059977904014]
  We propose a novel approach to identify meaningful and independent factors of variation in a dataset. Our method involves two separate latent subspaces for the target property and the remaining input information. We demonstrate on synthetic and molecular data that our approach identifies more meaningful factors which lead to sparser and more interpretable models.
  arXiv  Detail & Related papers  (2021-11-25T17:33:12Z)
• Marginalizable Density Models [14.50261153230204]
  We present a novel deep network architecture which provides closed form expressions for the probabilities, marginals and conditionals of any subset of the variables. The model also allows for parallelized sampling with only a logarithmic dependence of the time complexity on the number of variables.
  arXiv  Detail & Related papers  (2021-06-08T23:54:48Z)
• Disentangling Observed Causal Effects from Latent Confounders using Method of Moments [67.27068846108047]
  We provide guarantees on identifiability and learnability under mild assumptions. We develop efficient algorithms based on coupled tensor decomposition with linear constraints to obtain scalable and guaranteed solutions.
  arXiv  Detail & Related papers  (2021-01-17T07:48:45Z)
• Learning Disentangled Representations with Latent Variation Predictability [102.4163768995288]
  This paper defines the variation predictability of latent disentangled representations. Within an adversarial generation process, we encourage variation predictability by maximizing the mutual information between latent variations and corresponding image pairs. We develop an evaluation metric that does not rely on the ground-truth generative factors to measure the disentanglement of latent representations.
  arXiv  Detail & Related papers  (2020-07-25T08:54:26Z)
• not-MIWAE: Deep Generative Modelling with Missing not at Random Data [21.977065542645082]
  We present an approach for building and fitting deep latent variable models (DLVMs) in cases where the missing process is dependent on the missing data. Specifically, a deep neural network enables us to flexibly model the conditional distribution of the missingness pattern given the data. We show on various kinds of data sets and missingness patterns that explicitly modelling the missing process can be invaluable.
  arXiv  Detail & Related papers  (2020-06-23T10:06:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.