Toward Spatially Unbiased Generative Models

• URL: http://arxiv.org/abs/2108.01285v1
• Date: Tue, 3 Aug 2021 04:13:03 GMT
• Title: Toward Spatially Unbiased Generative Models
• Authors: Jooyoung Choi, Jungbeom Lee, Yonghyun Jeong, Sungroh Yoon
• Abstract summary: Recent image generation models show remarkable generation performance. However, they mirror strong location preference in datasets, which we call spatial bias. We argue that the generators rely on their implicit positional encoding to render spatial content.
• Score: 19.269719158344508
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent image generation models show remarkable generation performance. However, they mirror strong location preference in datasets, which we call spatial bias. Therefore, generators render poor samples at unseen locations and scales. We argue that the generators rely on their implicit positional encoding to render spatial content. From our observations, the generator's implicit positional encoding is translation-variant, making the generator spatially biased. To address this issue, we propose injecting explicit positional encoding at each scale of the generator. By learning the spatially unbiased generator, we facilitate the robust use of generators in multiple tasks, such as GAN inversion, multi-scale generation, generation of arbitrary sizes and aspect ratios. Furthermore, we show that our method can also be applied to denoising diffusion probabilistic models.

Related papers

• Improving Out-of-Distribution Robustness of Classifiers via Generative Interpolation [56.620403243640396]
  Deep neural networks achieve superior performance for learning from independent and identically distributed (i.i.d.) data. However, their performance deteriorates significantly when handling out-of-distribution (OoD) data. We develop a simple yet effective method called Generative Interpolation to fuse generative models trained from multiple domains for synthesizing diverse OoD samples.
  arXiv  Detail & Related papers  (2023-07-23T03:53:53Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• Self-Conditioned Generative Adversarial Networks for Image Editing [61.50205580051405]
  Generative Adversarial Networks (GANs) are susceptible to bias, learned from either the unbalanced data, or through mode collapse. We argue that this bias is responsible not only for fairness concerns, but that it plays a key role in the collapse of latent-traversal editing methods when deviating away from the distribution's core.
  arXiv  Detail & Related papers  (2022-02-08T18:08:24Z)
• AE-StyleGAN: Improved Training of Style-Based Auto-Encoders [21.51697087024866]
  StyleGANs have shown impressive results on data generation and manipulation in recent years. In this paper, we focus on style-based generators asking a scientific question: Does forcing such a generator to reconstruct real data lead to more disentangled latent space and make the inversion process from image to latent space easy? We describe a new methodology to train a style-based autoencoder where the encoder and generator are optimized end-to-end.
  arXiv  Detail & Related papers  (2021-10-17T04:25:51Z)
• Slimmable Generative Adversarial Networks [54.61774365777226]
  Generative adversarial networks (GANs) have achieved remarkable progress in recent years, but the continuously growing scale of models makes them challenging to deploy widely in practical applications. In this paper, we introduce slimmable GANs, which can flexibly switch the width of the generator to accommodate various quality-efficiency trade-offs at runtime.
  arXiv  Detail & Related papers  (2020-12-10T13:35:22Z)
• Positional Encoding as Spatial Inductive Bias in GANs [97.6622154941448]
  SinGAN shows impressive capability in learning internal patch distribution despite its limited effective receptive field. In this work, we show that such capability, to a large extent, is brought by the implicit positional encoding when using zero padding in the generators. We propose a new multi-scale training strategy and demonstrate its effectiveness in the state-of-the-art unconditional generator StyleGAN2.
  arXiv  Detail & Related papers  (2020-12-09T18:27:16Z)
• Mode Penalty Generative Adversarial Network with adapted Auto-encoder [0.15229257192293197]
  We propose a mode penalty GAN combined with pre-trained auto encoder for explicit representation of generated and real data samples in encoded space. We demonstrate that applying the proposed method to GANs helps generator's optimization becoming more stable and having faster convergence through experimental evaluations.
  arXiv  Detail & Related papers  (2020-11-16T03:39:53Z)
• Generative models with kernel distance in data space [10.002379593718471]
  LCW generator resembles a classical GAN in transforming Gaussian noise into data space. First, an autoencoder based architecture, using kernel measures, is built to model a manifold of data. We propose a Latent Trick mapping a Gaussian to latent in order to get the final model.
  arXiv  Detail & Related papers  (2020-09-15T19:11:47Z)
• Improving Inversion and Generation Diversity in StyleGAN using a Gaussianized Latent Space [41.20193123974535]
  Modern Generative Adversarial Networks are capable of creating artificial, photorealistic images from latent vectors living in a low-dimensional learned latent space. We show that, under a simple nonlinear operation, the data distribution can be modeled as Gaussian and therefore expressed using sufficient statistics. The resulting projections lie in smoother and better behaved regions of the latent space, as shown using performance for both real and generated images.
  arXiv  Detail & Related papers  (2020-09-14T15:45:58Z)
• Discriminator Contrastive Divergence: Semi-Amortized Generative Modeling by Exploring Energy of the Discriminator [85.68825725223873]
  Generative Adversarial Networks (GANs) have shown great promise in modeling high dimensional data. We introduce the Discriminator Contrastive Divergence, which is well motivated by the property of WGAN's discriminator. We demonstrate the benefits of significant improved generation on both synthetic data and several real-world image generation benchmarks.
  arXiv  Detail & Related papers  (2020-04-05T01:50:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.