GMM-Based Generative Adversarial Encoder Learning
- URL: http://arxiv.org/abs/2012.04525v1
- Date: Tue, 8 Dec 2020 16:12:16 GMT
- Title: GMM-Based Generative Adversarial Encoder Learning
- Authors: Yuri Feigin and Hedva Spitzer and Raja Giryes
- Abstract summary: We present a simple architectural setup that combines the generative capabilities of GAN with an encoder.
We model the output of the encoder latent space via a GMM, which leads to both good clustering using this latent space and improved image generation by the GAN.
- Score: 39.57350884615545
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: While GAN is a powerful model for generating images, its inability to infer a
latent space directly limits its use in applications requiring an encoder. Our
paper presents a simple architectural setup that combines the generative
capabilities of GAN with an encoder. We accomplish this by combining the
encoder with the discriminator using shared weights, then training them
simultaneously using a new loss term. We model the output of the encoder latent
space via a GMM, which leads to both good clustering using this latent space
and improved image generation by the GAN. Our framework is generic and can be
easily plugged into any GAN strategy. In particular, we demonstrate it both
with Vanilla GAN and Wasserstein GAN, where in both it leads to an improvement
in the generated images in terms of both the IS and FID scores. Moreover, we
show that our encoder learns a meaningful representation as its clustering
results are competitive with the current GAN-based state-of-the-art in
clustering.
Related papers
- In-Domain GAN Inversion for Faithful Reconstruction and Editability [132.68255553099834]
We propose in-domain GAN inversion, which consists of a domain-guided domain-regularized and a encoder to regularize the inverted code in the native latent space of the pre-trained GAN model.
We make comprehensive analyses on the effects of the encoder structure, the starting inversion point, as well as the inversion parameter space, and observe the trade-off between the reconstruction quality and the editing property.
arXiv Detail & Related papers (2023-09-25T08:42:06Z) - Complexity Matters: Rethinking the Latent Space for Generative Modeling [65.64763873078114]
In generative modeling, numerous successful approaches leverage a low-dimensional latent space, e.g., Stable Diffusion.
In this study, we aim to shed light on this under-explored topic by rethinking the latent space from the perspective of model complexity.
arXiv Detail & Related papers (2023-07-17T07:12:29Z) - LD-GAN: Low-Dimensional Generative Adversarial Network for Spectral
Image Generation with Variance Regularization [72.4394510913927]
Deep learning methods are state-of-the-art for spectral image (SI) computational tasks.
GANs enable diverse augmentation by learning and sampling from the data distribution.
GAN-based SI generation is challenging since the high-dimensionality nature of this kind of data hinders the convergence of the GAN training yielding to suboptimal generation.
We propose a statistical regularization to control the low-dimensional representation variance for the autoencoder training and to achieve high diversity of samples generated with the GAN.
arXiv Detail & Related papers (2023-04-29T00:25:02Z) - GIU-GANs: Global Information Utilization for Generative Adversarial
Networks [3.3945834638760948]
In this paper, we propose a new GANs called Involution Generative Adversarial Networks (GIU-GANs)
GIU-GANs leverages a brand new module called the Global Information Utilization (GIU) module, which integrates Squeeze-and-Excitation Networks (SENet) and involution.
Batch Normalization(BN) inevitably ignores the representation differences among noise sampled by the generator, and thus degrades the generated image quality.
arXiv Detail & Related papers (2022-01-25T17:17:15Z) - Information-theoretic stochastic contrastive conditional GAN:
InfoSCC-GAN [6.201770337181472]
We present a contrastive conditional generative adversarial network (Info SCC-GAN) with an explorable latent space.
Info SCC-GAN is derived based on an information-theoretic formulation of mutual information between input data and latent space representation.
Experiments show that Info SCC-GAN outperforms the "vanilla" EigenGAN in the image generation on AFHQ and CelebA datasets.
arXiv Detail & Related papers (2021-12-17T17:56:30Z) - DGL-GAN: Discriminator Guided Learning for GAN Compression [57.6150859067392]
Generative Adversarial Networks (GANs) with high computation costs have achieved remarkable results in synthesizing high-resolution images from random noise.
We propose a novel yet simple bf Discriminator bf Guided bf Learning approach for compressing vanilla bf GAN, dubbed bf DGL-GAN.
arXiv Detail & Related papers (2021-12-13T09:24:45Z) - DO-GAN: A Double Oracle Framework for Generative Adversarial Networks [28.904057977044374]
We propose a new approach to train Generative Adversarial Networks (GANs)
We deploy a double-oracle framework using the generator and discriminator oracles.
We apply our framework to established GAN architectures such as vanilla GAN, Deep Convolutional GAN, Spectral Normalization GAN and Stacked GAN.
arXiv Detail & Related papers (2021-02-17T05:11:18Z) - Positional Encoding as Spatial Inductive Bias in GANs [97.6622154941448]
SinGAN shows impressive capability in learning internal patch distribution despite its limited effective receptive field.
In this work, we show that such capability, to a large extent, is brought by the implicit positional encoding when using zero padding in the generators.
We propose a new multi-scale training strategy and demonstrate its effectiveness in the state-of-the-art unconditional generator StyleGAN2.
arXiv Detail & Related papers (2020-12-09T18:27:16Z) - Generate High Resolution Images With Generative Variational Autoencoder [0.0]
We present a novel neural network to generate high resolution images.
We replace the decoder of VAE with a discriminator while using the encoder as it is.
We evaluate our network on 3 different datasets: MNIST, LSUN and CelebA dataset.
arXiv Detail & Related papers (2020-08-12T20:15:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.