Statistical Regeneration Guarantees of the Wasserstein Autoencoder with Latent Space Consistency

• URL: http://arxiv.org/abs/2110.03995v1
• Date: Fri, 8 Oct 2021 09:26:54 GMT
• Title: Statistical Regeneration Guarantees of the Wasserstein Autoencoder with Latent Space Consistency
• Authors: Anish Chakrabarty and Swagatam Das
• Abstract summary: We investigate the statistical properties of Wasserstein Autoencoder (WAE) We provide statistical guarantees that WAE achieves the target distribution in the latent space. This study hints at the class of distributions WAE can reconstruct after suffering a compression in the form of a latent law.
• Score: 14.07437185521097
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The introduction of Variational Autoencoders (VAE) has been marked as a breakthrough in the history of representation learning models. Besides having several accolades of its own, VAE has successfully flagged off a series of inventions in the form of its immediate successors. Wasserstein Autoencoder (WAE), being an heir to that realm carries with it all of the goodness and heightened generative promises, matching even the generative adversarial networks (GANs). Needless to say, recent years have witnessed a remarkable resurgence in statistical analyses of the GANs. Similar examinations for Autoencoders, however, despite their diverse applicability and notable empirical performance, remain largely absent. To close this gap, in this paper, we investigate the statistical properties of WAE. Firstly, we provide statistical guarantees that WAE achieves the target distribution in the latent space, utilizing the Vapnik Chervonenkis (VC) theory. The main result, consequently ensures the regeneration of the input distribution, harnessing the potential offered by Optimal Transport of measures under the Wasserstein metric. This study, in turn, hints at the class of distributions WAE can reconstruct after suffering a compression in the form of a latent law.

Related papers

• Variational Rank Reduction Autoencoder [1.3980986259786223]
  We present Variational Rank Reduction Autoencoders (VRRAEs) a model that leverages the advantages of both RRAEs and VAEs.<n>Our results include a synthetic dataset of a small size that showcases the robustness of VRRAEs against collapse, and three real-world datasets.
  arXiv  Detail & Related papers  (2025-05-14T15:08:28Z)
• Continuous Visual Autoregressive Generation via Score Maximization [69.67438563485887]
  We introduce a Continuous VAR framework that enables direct visual autoregressive generation without vector quantization.<n>Within this framework, all we need is to select a strictly proper score and set it as the training objective to optimize.
  arXiv  Detail & Related papers  (2025-05-12T17:58:14Z)
• Concurrent Density Estimation with Wasserstein Autoencoders: Some Statistical Insights [20.894503281724052]
  Wasserstein Autoencoders (WAEs) have been a pioneering force in the realm of deep generative models. Our work is an attempt to offer a theoretical understanding of the machinery behind WAEs.
  arXiv  Detail & Related papers  (2023-12-11T18:27:25Z)
• Statistical Guarantees for Variational Autoencoders using PAC-Bayesian Theory [2.828173677501078]
  Variational Autoencoders (VAEs) have become central in machine learning. This work develops statistical guarantees for VAEs using PAC-Bayesian theory.
  arXiv  Detail & Related papers  (2023-10-07T22:35:26Z)
• Symmetric Equilibrium Learning of VAEs [56.56929742714685]
  We view variational autoencoders (VAEs) as decoder-encoder pairs, which map distributions in the data space to distributions in the latent space and vice versa. We propose a Nash equilibrium learning approach, which is symmetric with respect to the encoder and decoder and allows learning VAEs in situations where both the data and the latent distributions are accessible only by sampling.
  arXiv  Detail & Related papers  (2023-07-19T10:27:34Z)
• Exact Non-Oblivious Performance of Rademacher Random Embeddings [79.28094304325116]
  This paper revisits the performance of Rademacher random projections. It establishes novel statistical guarantees that are numerically sharp and non-oblivious with respect to the input data.
  arXiv  Detail & Related papers  (2023-03-21T11:45:27Z)
• Be Your Own Neighborhood: Detecting Adversarial Example by the Neighborhood Relations Built on Self-Supervised Learning [64.78972193105443]
  This paper presents a novel AE detection framework, named trustworthy for predictions. performs the detection by distinguishing the AE's abnormal relation with its augmented versions. An off-the-shelf Self-Supervised Learning (SSL) model is used to extract the representation and predict the label.
  arXiv  Detail & Related papers  (2022-08-31T08:18:44Z)
• Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [61.827054365139645]
  Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. We propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space.
  arXiv  Detail & Related papers  (2021-10-24T07:58:13Z)
• Momentum Contrastive Autoencoder: Using Contrastive Learning for Latent Space Distribution Matching in WAE [51.09507030387935]
  Wasserstein autoencoder (WAE) shows that matching two distributions is equivalent to minimizing a simple autoencoder (AE) loss under the constraint that the latent space of this AE matches a pre-specified prior distribution. We propose to use the contrastive learning framework that has been shown to be effective for self-supervised representation learning, as a means to resolve this problem. We show that using the contrastive learning framework to optimize the WAE loss achieves faster convergence and more stable optimization compared with existing popular algorithms for WAE.
  arXiv  Detail & Related papers  (2021-10-19T22:55:47Z)
• Generalizing Variational Autoencoders with Hierarchical Empirical Bayes [6.273154057349038]
  We present Hierarchical Empirical Bayes Autoencoder (HEBAE), a computationally stable framework for probabilistic generative models. Our key contributions are two-fold. First, we make gains by placing a hierarchical prior over the encoding distribution, enabling us to adaptively balance the trade-off between minimizing the reconstruction loss function and avoiding over-regularization.
  arXiv  Detail & Related papers  (2020-07-20T18:18:39Z)
• To Regularize or Not To Regularize? The Bias Variance Trade-off in Regularized AEs [10.611727286504994]
  We study the effect of the latent prior on the generation deterministic quality of AE models. We show that our model, called FlexAE, is the new state-of-the-art for the AE based generative models.
  arXiv  Detail & Related papers  (2020-06-10T14:00:14Z)
• Distribution Approximation and Statistical Estimation Guarantees of Generative Adversarial Networks [82.61546580149427]
  Generative Adversarial Networks (GANs) have achieved a great success in unsupervised learning. This paper provides approximation and statistical guarantees of GANs for the estimation of data distributions with densities in a H"older space.
  arXiv  Detail & Related papers  (2020-02-10T16:47:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.