Sampling From Autoencoders' Latent Space via Quantization And Probability Mass Function Concepts

• URL: http://arxiv.org/abs/2308.10704v1
• Date: Mon, 21 Aug 2023 13:18:12 GMT
• Title: Sampling From Autoencoders' Latent Space via Quantization And Probability Mass Function Concepts
• Authors: Aymene Mohammed Bouayed and Adrian Iaccovelli and David Naccache
• Abstract summary: We introduce a novel post-training sampling algorithm rooted in the concept of probability mass functions, coupled with a quantization process. Our proposed algorithm establishes a vicinity around each latent vector from the input data and then proceeds to draw samples from these defined neighborhoods. This strategic approach ensures that the sampled latent vectors predominantly inhabit high-probability regions, which, in turn, can be effectively transformed into authentic real-world images.
• Score: 1.534667887016089
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: In this study, we focus on sampling from the latent space of generative models built upon autoencoders so as the reconstructed samples are lifelike images. To do to, we introduce a novel post-training sampling algorithm rooted in the concept of probability mass functions, coupled with a quantization process. Our proposed algorithm establishes a vicinity around each latent vector from the input data and then proceeds to draw samples from these defined neighborhoods. This strategic approach ensures that the sampled latent vectors predominantly inhabit high-probability regions, which, in turn, can be effectively transformed into authentic real-world images. A noteworthy point of comparison for our sampling algorithm is the sampling technique based on Gaussian mixture models (GMM), owing to its inherent capability to represent clusters. Remarkably, we manage to improve the time complexity from the previous $\mathcal{O}(n\times d \times k \times i)$ associated with GMM sampling to a much more streamlined $\mathcal{O}(n\times d)$, thereby resulting in substantial speedup during runtime. Moreover, our experimental results, gauged through the Fr\'echet inception distance (FID) for image generation, underscore the superior performance of our sampling algorithm across a diverse range of models and datasets. On the MNIST benchmark dataset, our approach outperforms GMM sampling by yielding a noteworthy improvement of up to $0.89$ in FID value. Furthermore, when it comes to generating images of faces and ocular images, our approach showcases substantial enhancements with FID improvements of $1.69$ and $0.87$ respectively, as compared to GMM sampling, as evidenced on the CelebA and MOBIUS datasets. Lastly, we substantiate our methodology's efficacy in estimating latent space distributions in contrast to GMM sampling, particularly through the lens of the Wasserstein distance.

Related papers

• Align Your Steps: Optimizing Sampling Schedules in Diffusion Models [63.927438959502226]
  Diffusion models (DMs) have established themselves as the state-of-the-art generative modeling approach in the visual domain and beyond. A crucial drawback of DMs is their slow sampling speed, relying on many sequential function evaluations through large neural networks. We propose a general and principled approach to optimizing the sampling schedules of DMs for high-quality outputs.
  arXiv  Detail & Related papers  (2024-04-22T18:18:41Z)
• Deep Gaussian mixture model for unsupervised image segmentation [1.3654846342364308]
  In many tasks sufficient pixel-level labels are very difficult to obtain. We propose a method which combines a Gaussian mixture model (GMM) with unsupervised deep learning techniques. We demonstrate the advantages of our method in various experiments on the example of infarct segmentation on multi-sequence MRI images.
  arXiv  Detail & Related papers  (2024-04-18T15:20:59Z)
• Deep Generative Sampling in the Dual Divergence Space: A Data-efficient & Interpretative Approach for Generative AI [29.13807697733638]
  We build on the remarkable achievements in generative sampling of natural images. We propose an innovative challenge, potentially overly ambitious, which involves generating samples that resemble images. The statistical challenge lies in the small sample size, sometimes consisting of a few hundred subjects.
  arXiv  Detail & Related papers  (2024-04-10T22:35:06Z)
• Iterated Denoising Energy Matching for Sampling from Boltzmann Densities [109.23137009609519]
  Iterated Denoising Energy Matching (iDEM) iDEM alternates between (I) sampling regions of high model density from a diffusion-based sampler and (II) using these samples in our matching objective. We show that the proposed approach achieves state-of-the-art performance on all metrics and trains $2-5times$ faster.
  arXiv  Detail & Related papers  (2024-02-09T01:11:23Z)
• MB-RACS: Measurement-Bounds-based Rate-Adaptive Image Compressed Sensing Network [65.1004435124796]
  We propose a Measurement-Bounds-based Rate-Adaptive Image Compressed Sensing Network (MB-RACS) framework. Our experiments demonstrate that the proposed MB-RACS method surpasses current leading methods.
  arXiv  Detail & Related papers  (2024-01-19T04:40:20Z)
• Improved DDIM Sampling with Moment Matching Gaussian Mixtures [1.450405446885067]
  We propose using a Gaussian Mixture Model (GMM) as reverse transition operator ( kernel) within the Denoising Diffusion Implicit Models (DDIM) framework. We match the first and second order central moments of the DDPM forward marginals by constraining the parameters of the GMM. Our results suggest that using the GMM kernel leads to significant improvements in the quality of the generated samples when the number of sampling steps is small.
  arXiv  Detail & Related papers  (2023-11-08T00:24:50Z)
• Incremental Multimodal Surface Mapping via Self-Organizing Gaussian Mixture Models [1.0878040851638]
  This letter describes an incremental multimodal surface mapping methodology, which represents the environment as a continuous probabilistic model. The strategy employed in this work utilizes Gaussian mixture models (GMMs) to represent the environment. To bridge this gap, this letter introduces a spatial hash map for rapid GMM submap extraction combined with an approach to determine relevant and redundant data in a point cloud.
  arXiv  Detail & Related papers  (2023-09-19T19:49:03Z)
• Constrained Probabilistic Mask Learning for Task-specific Undersampled MRI Reconstruction [8.44194619347218]
  Undersampling is a common method in Magnetic Resonance Imaging (MRI) to subsample the number of data points in k-space. We propose a method that directly learns the undersampling masks from data points. We show that different anatomic regions reveal distinct optimal undersampling masks.
  arXiv  Detail & Related papers  (2023-05-25T14:42:04Z)
• A new perspective on probabilistic image modeling [92.89846887298852]
  We present a new probabilistic approach for image modeling capable of density estimation, sampling and tractable inference. DCGMMs can be trained end-to-end by SGD from random initial conditions, much like CNNs. We show that DCGMMs compare favorably to several recent PC and SPN models in terms of inference, classification and sampling.
  arXiv  Detail & Related papers  (2022-03-21T14:53:57Z)
• Learning Energy-Based Models by Diffusion Recovery Likelihood [61.069760183331745]
  We present a diffusion recovery likelihood method to tractably learn and sample from a sequence of energy-based models. After training, synthesized images can be generated by the sampling process that initializes from Gaussian white noise distribution. On unconditional CIFAR-10 our method achieves FID 9.58 and inception score 8.30, superior to the majority of GANs.
  arXiv  Detail & Related papers  (2020-12-15T07:09:02Z)
• A Generative Learning Approach for Spatio-temporal Modeling in Connected Vehicular Network [55.852401381113786]
  This paper proposes LaMI (Latency Model Inpainting), a novel framework to generate a comprehensive-temporal quality framework for wireless access latency of connected vehicles. LaMI adopts the idea from image inpainting and synthesizing and can reconstruct the missing latency samples by a two-step procedure. In particular, it first discovers the spatial correlation between samples collected in various regions using a patching-based approach and then feeds the original and highly correlated samples into a Varienational Autocoder (VAE)
  arXiv  Detail & Related papers  (2020-03-16T03:43:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.