Identifiable Variational Autoencoders via Sparse Decoding
- URL: http://arxiv.org/abs/2110.10804v1
- Date: Wed, 20 Oct 2021 22:11:33 GMT
- Title: Identifiable Variational Autoencoders via Sparse Decoding
- Authors: Gemma E. Moran, Dhanya Sridhar, Yixin Wang and David M. Blei
- Abstract summary: We develop the Sparse VAE, a deep generative model for unsupervised representation learning on high-dimensional data.
We first show that the Sparse VAE is identifiable: given data drawn from the model, there exists a uniquely optimal set of factors.
We empirically study the Sparse VAE with both simulated and real data.
- Score: 37.30831737046145
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We develop the Sparse VAE, a deep generative model for unsupervised
representation learning on high-dimensional data. Given a dataset of
observations, the Sparse VAE learns a set of latent factors that captures its
distribution. The model is sparse in the sense that each feature of the dataset
(i.e., each dimension) depends on a small subset of the latent factors. As
examples, in ratings data each movie is only described by a few genres; in text
data each word is only applicable to a few topics; in genomics, each gene is
active in only a few biological processes. We first show that the Sparse VAE is
identifiable: given data drawn from the model, there exists a uniquely optimal
set of factors. (In contrast, most VAE-based models are not identifiable.) The
key assumption behind Sparse-VAE identifiability is the existence of "anchor
features", where for each factor there exists a feature that depends only on
that factor. Importantly, the anchor features do not need to be known in
advance. We then show how to fit the Sparse VAE with variational EM. Finally,
we empirically study the Sparse VAE with both simulated and real data. We find
that it recovers meaningful latent factors and has smaller heldout
reconstruction error than related methods.
Related papers
- Predictive variational autoencoder for learning robust representations
of time-series data [0.0]
We propose a VAE architecture that predicts the next point in time and show that it mitigates the learning of spurious features.
We show that together these two constraints on VAEs to be smooth over time produce robust latent representations and faithfully recover latent factors on synthetic datasets.
arXiv Detail & Related papers (2023-12-12T02:06:50Z) - SepVAE: a contrastive VAE to separate pathological patterns from healthy ones [2.619659560375341]
Contrastive Analysis VAE (CA-VAEs) is a family of Variational auto-encoders (VAEs) that aims at separating the common factors of variation between a background dataset (BG) and a target dataset (TG)
We show a better performance than previous CA-VAEs methods on three medical applications and a natural images dataset (CelebA)
arXiv Detail & Related papers (2023-07-12T14:52:21Z) - A Self-Encoder for Learning Nearest Neighbors [5.297261090056809]
The self-encoder learns to distribute the data samples in the embedding space so that they are linearly separable from one another.
Unlike regular nearest neighbors, the predictions resulting from this encoding of data are invariant to any scaling of features.
arXiv Detail & Related papers (2023-06-25T14:30:31Z) - Posterior Collapse and Latent Variable Non-identifiability [54.842098835445]
We propose a class of latent-identifiable variational autoencoders, deep generative models which enforce identifiability without sacrificing flexibility.
Across synthetic and real datasets, latent-identifiable variational autoencoders outperform existing methods in mitigating posterior collapse and providing meaningful representations of the data.
arXiv Detail & Related papers (2023-01-02T06:16:56Z) - Equivariance Allows Handling Multiple Nuisance Variables When Analyzing
Pooled Neuroimaging Datasets [53.34152466646884]
In this paper, we show how bringing recent results on equivariant representation learning instantiated on structured spaces together with simple use of classical results on causal inference provides an effective practical solution.
We demonstrate how our model allows dealing with more than one nuisance variable under some assumptions and can enable analysis of pooled scientific datasets in scenarios that would otherwise entail removing a large portion of the samples.
arXiv Detail & Related papers (2022-03-29T04:54:06Z) - SubOmiEmbed: Self-supervised Representation Learning of Multi-omics Data
for Cancer Type Classification [4.992154875028543]
Integration and analysis of multi-omics data give us a broad view of tumours, which can improve clinical decision making.
SubOmiEmbed produces comparable results to the baseline OmiEmbed with a much smaller network and by using just a subset of the data.
This work can be improved to integrate mutation-based genomic data as well.
arXiv Detail & Related papers (2022-02-03T16:39:09Z) - Multivariate Data Explanation by Jumping Emerging Patterns Visualization [78.6363825307044]
We present VAX (multiVariate dAta eXplanation), a new VA method to support the identification and visual interpretation of patterns in multivariate data sets.
Unlike the existing similar approaches, VAX uses the concept of Jumping Emerging Patterns to identify and aggregate several diversified patterns, producing explanations through logic combinations of data variables.
arXiv Detail & Related papers (2021-06-21T13:49:44Z) - The Shapley Value of coalition of variables provides better explanations [0.0]
We show that Shapley Values (SV) are still poorly understood.
We show that the popular practice that consists in summing the SV of dummy variables is false.
We provide a correct computation and inference of important variables.
arXiv Detail & Related papers (2021-03-24T17:02:57Z) - CausalVAE: Structured Causal Disentanglement in Variational Autoencoder [52.139696854386976]
The framework of variational autoencoder (VAE) is commonly used to disentangle independent factors from observations.
We propose a new VAE based framework named CausalVAE, which includes a Causal Layer to transform independent factors into causal endogenous ones.
Results show that the causal representations learned by CausalVAE are semantically interpretable, and their causal relationship as a Directed Acyclic Graph (DAG) is identified with good accuracy.
arXiv Detail & Related papers (2020-04-18T20:09:34Z) - Parameter Space Factorization for Zero-Shot Learning across Tasks and
Languages [112.65994041398481]
We propose a Bayesian generative model for the space of neural parameters.
We infer the posteriors over such latent variables based on data from seen task-language combinations.
Our model yields comparable or better results than state-of-the-art, zero-shot cross-lingual transfer methods.
arXiv Detail & Related papers (2020-01-30T16:58:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.