Improving the Generation of VAEs with High Dimensional Latent Spaces by the use of Hyperspherical Coordinates

• URL: http://arxiv.org/abs/2507.15900v1
• Date: Mon, 21 Jul 2025 05:10:43 GMT
• Title: Improving the Generation of VAEs with High Dimensional Latent Spaces by the use of Hyperspherical Coordinates
• Authors: Alejandro Ascarate, Leo Lebrat, Rodrigo Santa Cruz, Clinton Fookes, Olivier Salvado,
• Abstract summary: Variational autoencoders (VAE) encode data into lower-dimensional latent vectors before decoding those vectors back to data.<n>We propose a new parameterization of the latent space with limited computational overhead.
• Score: 59.4526726541389
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Variational autoencoders (VAE) encode data into lower-dimensional latent vectors before decoding those vectors back to data. Once trained, decoding a random latent vector from the prior usually does not produce meaningful data, at least when the latent space has more than a dozen dimensions. In this paper, we investigate this issue by drawing insight from high dimensional statistics: in these regimes, the latent vectors of a standard VAE are by construction distributed uniformly on a hypersphere. We propose to formulate the latent variables of a VAE using hyperspherical coordinates, which allows compressing the latent vectors towards an island on the hypersphere, thereby reducing the latent sparsity and we show that this improves the generation ability of the VAE. We propose a new parameterization of the latent space with limited computational overhead.

Related papers

• LASERS: LAtent Space Encoding for Representations with Sparsity for Generative Modeling [3.9426000822656224]
  We show that our more latent space is more expressive and has leads to better representations than the Vector Quantization approach. Our results thus suggest that the true benefit of the VQ approach might not be from discretization of the latent space, but rather the lossy compression of the latent space.
  arXiv  Detail & Related papers  (2024-09-16T08:20:58Z)
• HyperVQ: MLR-based Vector Quantization in Hyperbolic Space [56.4245885674567]
  A common solution is to employ Vector Quantization (VQ) within VQ Variational Autoencoders (VQVAEs)<n>We introduce HyperVQ, a novel approach that formulates VQ as a hyperbolic Multinomial Logistic Regression (MLR) problem.<n>Our experiments demonstrate that HyperVQ matches traditional VQ in generative and reconstruction tasks, while surpassing it in discriminative performance.
  arXiv  Detail & Related papers  (2024-03-18T03:17:08Z)
• Distributional Reduction: Unifying Dimensionality Reduction and Clustering with Gromov-Wasserstein [56.62376364594194]
  Unsupervised learning aims to capture the underlying structure of potentially large and high-dimensional datasets.<n>In this work, we revisit these approaches under the lens of optimal transport and exhibit relationships with the Gromov-Wasserstein problem.<n>This unveils a new general framework, called distributional reduction, that recovers DR and clustering as special cases and allows addressing them jointly within a single optimization problem.
  arXiv  Detail & Related papers  (2024-02-03T19:00:19Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• Dimension-reduced KRnet maps for high-dimensional inverse problems [0.0]
  We present a dimension-reduced KRnet map approach (DR-KRnet) for high-dimensional inverse problems. Our approach consists of two main components: data-driven VAE prior and density approximation of the posterior of the latent variable.
  arXiv  Detail & Related papers  (2023-03-01T15:16:27Z)
• RENs: Relevance Encoding Networks [0.0]
  This paper proposes relevance encoding networks (RENs): a novel probabilistic VAE-based framework that uses the automatic relevance determination (ARD) prior in the latent space to learn the data-specific bottleneck dimensionality. We show that the proposed model learns the relevant latent bottleneck dimensionality without compromising the representation and generation quality of the samples.
  arXiv  Detail & Related papers  (2022-05-25T21:53:48Z)
• Robust Differentiable SVD [117.35644933471401]
  Eigendecomposition of symmetric matrices is at the heart of many computer vision algorithms. Instability arises in the presence of eigenvalues that are close to each other. We show that the Taylor expansion of the SVD gradient is theoretically equivalent to the gradient obtained using PI without relying on an iterative process.
  arXiv  Detail & Related papers  (2021-04-08T15:04:15Z)
• Variational Hyper-Encoding Networks [62.74164588885455]
  We propose a framework called HyperVAE for encoding distributions of neural network parameters theta. We predict the posterior distribution of the latent code, then use a matrix-network decoder to generate a posterior distribution q(theta)
  arXiv  Detail & Related papers  (2020-05-18T06:46:09Z)
• Dynamic Narrowing of VAE Bottlenecks Using GECO and L0 Regularization [5.57310999362848]
  We have developed a technique to shrink the latent space dimensionality of VAEs automatically and on-the-fly during training. This paper presents the algorithmic details of our method along with experimental results on five different datasets.
  arXiv  Detail & Related papers  (2020-03-24T14:58:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.