Geometry-Aware Hamiltonian Variational Auto-Encoder
- URL: http://arxiv.org/abs/2010.11518v1
- Date: Thu, 22 Oct 2020 08:26:46 GMT
- Title: Geometry-Aware Hamiltonian Variational Auto-Encoder
- Authors: Cl\'ement Chadebec (CRC, Universit\'e de Paris), Cl\'ement Mantoux
(ARAMIS) and St\'ephanie Allassonni\`ere (CRC, Universit\'e de Paris)
- Abstract summary: Variational auto-encoders (VAEs) have proven to be a well suited tool for performing dimensionality reduction by extracting latent variables lying in a potentially much smaller dimensional space than the data.
However, such generative models may perform poorly when trained on small data sets which are abundant in many real-life fields such as medicine.
We argue that such latent space modelling provides useful information about its underlying structure leading to far more meaningfuls, more realistic data-generation and more reliable clustering.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Variational auto-encoders (VAEs) have proven to be a well suited tool for
performing dimensionality reduction by extracting latent variables lying in a
potentially much smaller dimensional space than the data. Their ability to
capture meaningful information from the data can be easily apprehended when
considering their capability to generate new realistic samples or perform
potentially meaningful interpolations in a much smaller space. However, such
generative models may perform poorly when trained on small data sets which are
abundant in many real-life fields such as medicine. This may, among others,
come from the lack of structure of the latent space, the geometry of which is
often under-considered. We thus propose in this paper to see the latent space
as a Riemannian manifold endowed with a parametrized metric learned at the same
time as the encoder and decoder networks. This metric is then used in what we
called the Riemannian Hamiltonian VAE which extends the Hamiltonian VAE
introduced by arXiv:1805.11328 to better exploit the underlying geometry of the
latent space. We argue that such latent space modelling provides useful
information about its underlying structure leading to far more meaningful
interpolations, more realistic data-generation and more reliable clustering.
Related papers
- (Deep) Generative Geodesics [57.635187092922976]
We introduce a newian metric to assess the similarity between any two data points.
Our metric leads to the conceptual definition of generative distances and generative geodesics.
Their approximations are proven to converge to their true values under mild conditions.
arXiv Detail & Related papers (2024-07-15T21:14:02Z) - Distributional Reduction: Unifying Dimensionality Reduction and Clustering with Gromov-Wasserstein [56.62376364594194]
Unsupervised learning aims to capture the underlying structure of potentially large and high-dimensional datasets.
In this work, we revisit these approaches under the lens of optimal transport and exhibit relationships with the Gromov-Wasserstein problem.
This unveils a new general framework, called distributional reduction, that recovers DR and clustering as special cases and allows addressing them jointly within a single optimization problem.
arXiv Detail & Related papers (2024-02-03T19:00:19Z) - Disentanglement via Latent Quantization [60.37109712033694]
In this work, we construct an inductive bias towards encoding to and decoding from an organized latent space.
We demonstrate the broad applicability of this approach by adding it to both basic data-re (vanilla autoencoder) and latent-reconstructing (InfoGAN) generative models.
arXiv Detail & Related papers (2023-05-28T06:30:29Z) - VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables.
The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning.
We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
arXiv Detail & Related papers (2023-04-03T13:13:19Z) - Relative representations enable zero-shot latent space communication [19.144630518400604]
Neural networks embed the geometric structure of a data manifold lying in a high-dimensional space into latent representations.
We show how neural architectures can leverage these relative representations to guarantee, in practice, latent isometry invariance.
arXiv Detail & Related papers (2022-09-30T12:37:03Z) - Semi-Supervised Manifold Learning with Complexity Decoupled Chart
Autoencoders [65.2511270059236]
This work introduces a chart autoencoder with an asymmetric encoding-decoding process that can incorporate additional semi-supervised information such as class labels.
We discuss the theoretical approximation power of such networks that essentially depends on the intrinsic dimension of the data manifold.
arXiv Detail & Related papers (2022-08-22T19:58:03Z) - Intrinsic dimension estimation for discrete metrics [65.5438227932088]
In this letter we introduce an algorithm to infer the intrinsic dimension (ID) of datasets embedded in discrete spaces.
We demonstrate its accuracy on benchmark datasets, and we apply it to analyze a metagenomic dataset for species fingerprinting.
This suggests that evolutive pressure acts on a low-dimensional manifold despite the high-dimensionality of sequences' space.
arXiv Detail & Related papers (2022-07-20T06:38:36Z) - RENs: Relevance Encoding Networks [0.0]
This paper proposes relevance encoding networks (RENs): a novel probabilistic VAE-based framework that uses the automatic relevance determination (ARD) prior in the latent space to learn the data-specific bottleneck dimensionality.
We show that the proposed model learns the relevant latent bottleneck dimensionality without compromising the representation and generation quality of the samples.
arXiv Detail & Related papers (2022-05-25T21:53:48Z) - Characterizing the Latent Space of Molecular Deep Generative Models with
Persistent Homology Metrics [21.95240820041655]
Variational Autos (VAEs) are generative models in which encoder-decoder network pairs are trained to reconstruct training data distributions.
We propose a method for measuring how well the latent space of deep generative models is able to encode structural and chemical features.
arXiv Detail & Related papers (2020-10-18T13:33:02Z) - Variational Autoencoder with Learned Latent Structure [4.41370484305827]
We introduce the Variational Autoencoder with Learned Latent Structure (VAELLS)
VAELLS incorporates a learnable manifold model into the latent space of a VAE.
We validate our model on examples with known latent structure and also demonstrate its capabilities on a real-world dataset.
arXiv Detail & Related papers (2020-06-18T14:59:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.