InteL-VAEs: Adding Inductive Biases to Variational Auto-Encoders via
Intermediary Latents
- URL: http://arxiv.org/abs/2106.13746v1
- Date: Fri, 25 Jun 2021 16:34:05 GMT
- Title: InteL-VAEs: Adding Inductive Biases to Variational Auto-Encoders via
Intermediary Latents
- Authors: Ning Miao, Emile Mathieu, N. Siddharth, Yee Whye Teh, Tom Rainforth
- Abstract summary: We introduce a simple and effective method for learning VAEs with controllable biases by using an intermediary set of latent variables.
In particular, it allows us to impose desired properties like sparsity or clustering on learned representations.
We show that this, in turn, allows InteL-VAEs to learn both better generative models and representations.
- Score: 60.785317191131284
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We introduce a simple and effective method for learning VAEs with
controllable inductive biases by using an intermediary set of latent variables.
This allows us to overcome the limitations of the standard Gaussian prior
assumption. In particular, it allows us to impose desired properties like
sparsity or clustering on learned representations, and incorporate prior
information into the learned model. Our approach, which we refer to as the
Intermediary Latent Space VAE (InteL-VAE), is based around controlling the
stochasticity of the encoding process with the intermediary latent variables,
before deterministically mapping them forward to our target latent
representation, from which reconstruction is performed. This allows us to
maintain all the advantages of the traditional VAE framework, while
incorporating desired prior information, inductive biases, and even topological
information through the latent mapping. We show that this, in turn, allows
InteL-VAEs to learn both better generative models and representations.
Related papers
- Symmetric Equilibrium Learning of VAEs [56.56929742714685]
We view variational autoencoders (VAEs) as decoder-encoder pairs, which map distributions in the data space to distributions in the latent space and vice versa.
We propose a Nash equilibrium learning approach, which is symmetric with respect to the encoder and decoder and allows learning VAEs in situations where both the data and the latent distributions are accessible only by sampling.
arXiv Detail & Related papers (2023-07-19T10:27:34Z) - Disentanglement via Latent Quantization [60.37109712033694]
In this work, we construct an inductive bias towards encoding to and decoding from an organized latent space.
We demonstrate the broad applicability of this approach by adding it to both basic data-re (vanilla autoencoder) and latent-reconstructing (InfoGAN) generative models.
arXiv Detail & Related papers (2023-05-28T06:30:29Z) - RevUp: Revise and Update Information Bottleneck for Event Representation [16.54912614895861]
In machine learning, latent variables play a key role to capture the underlying structure of data, but they are often unsupervised.
We propose a semi-supervised information bottleneck-based model that enables the use of side knowledge to direct the learning of discrete latent variables.
We show that our approach generalizes an existing method of parameter injection, and perform an empirical case study of our approach on language-based event modeling.
arXiv Detail & Related papers (2022-05-24T17:54:59Z) - Covariate-informed Representation Learning with Samplewise Optimal
Identifiable Variational Autoencoders [15.254297587065595]
Recently proposed identifiable variational autoencoder (iVAE) provides a promising approach for learning latent independent components of the data.
We develop a new approach, co-informed identifiable VAE (CI-iVAE)
In doing so, the objective function enforces the inverse relation, and learned representation contains more information of observations.
arXiv Detail & Related papers (2022-02-09T00:18:33Z) - Regularizing Variational Autoencoder with Diversity and Uncertainty
Awareness [61.827054365139645]
Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference.
We propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space.
arXiv Detail & Related papers (2021-10-24T07:58:13Z) - Closed-Form Factorization of Latent Semantics in GANs [65.42778970898534]
A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images.
In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner.
We propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
arXiv Detail & Related papers (2020-07-13T18:05:36Z) - Variational Mutual Information Maximization Framework for VAE Latent
Codes with Continuous and Discrete Priors [5.317548969642376]
Variational Autoencoder (VAE) is a scalable method for learning directed latent variable models of complex data.
We propose Variational Mutual Information Maximization Framework for VAE to address this issue.
arXiv Detail & Related papers (2020-06-02T09:05:51Z) - Guided Variational Autoencoder for Disentanglement Learning [79.02010588207416]
We propose an algorithm, guided variational autoencoder (Guided-VAE), that is able to learn a controllable generative model by performing latent representation disentanglement learning.
We design an unsupervised strategy and a supervised strategy in Guided-VAE and observe enhanced modeling and controlling capability over the vanilla VAE.
arXiv Detail & Related papers (2020-04-02T20:49:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.