Guiding GANs: How to control non-conditional pre-trained GANs for conditional image generation

• URL: http://arxiv.org/abs/2101.00990v1
• Date: Mon, 4 Jan 2021 14:03:32 GMT
• Title: Guiding GANs: How to control non-conditional pre-trained GANs for conditional image generation
• Authors: Manel Mateos, Alejandro Gonz\'alez, Xavier Sevillano
• Abstract summary: We present a novel method for guiding generic non-conditional GANs to behave as conditional GANs. Our approach adds into the mix an encoder network to generate the high-dimensional random input that are fed to the generator network of a non-conditional GAN.
• Score: 69.10717733870575
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative Adversarial Networks (GANs) are an arrange of two neural networks -- the generator and the discriminator -- that are jointly trained to generate artificial data, such as images, from random inputs. The quality of these generated images has recently reached such levels that can often lead both machines and humans into mistaking fake for real examples. However, the process performed by the generator of the GAN has some limitations when we want to condition the network to generate images from subcategories of a specific class. Some recent approaches tackle this \textit{conditional generation} by introducing extra information prior to the training process, such as image semantic segmentation or textual descriptions. While successful, these techniques still require defining beforehand the desired subcategories and collecting large labeled image datasets representing them to train the GAN from scratch. In this paper we present a novel and alternative method for guiding generic non-conditional GANs to behave as conditional GANs. Instead of re-training the GAN, our approach adds into the mix an encoder network to generate the high-dimensional random input vectors that are fed to the generator network of a non-conditional GAN to make it generate images from a specific subcategory. In our experiments, when compared to training a conditional GAN from scratch, our guided GAN is able to generate artificial images of perceived quality comparable to that of non-conditional GANs after training the encoder on just a few hundreds of images, which substantially accelerates the process and enables adding new subcategories seamlessly.

Related papers

• Unlocking Pre-trained Image Backbones for Semantic Image Synthesis [29.688029979801577]
  We propose a new class of GAN discriminators for semantic image synthesis that generates highly realistic images. Our model, which we dub DP-SIMS, achieves state-of-the-art results in terms of image quality and consistency with the input label maps on ADE-20K, COCO-Stuff, and Cityscapes.
  arXiv  Detail & Related papers  (2023-12-20T09:39:19Z)
• Spider GAN: Leveraging Friendly Neighbors to Accelerate GAN Training [20.03447539784024]
  We propose a novel approach for training GANs with images as inputs, but without enforcing any pairwise constraints. The process can be made efficient by identifying closely related datasets, or a friendly neighborhood'' of the target distribution. We show that the Spider GAN formulation results in faster convergence, as the generator can discover correspondence even between seemingly unrelated datasets.
  arXiv  Detail & Related papers  (2023-05-12T17:03:18Z)
• Traditional Classification Neural Networks are Good Generators: They are Competitive with DDPMs and GANs [104.72108627191041]
  We show that conventional neural network classifiers can generate high-quality images comparable to state-of-the-art generative models. We propose a mask-based reconstruction module to make semantic gradients-aware to synthesize plausible images. We show that our method is also applicable to text-to-image generation by regarding image-text foundation models.
  arXiv  Detail & Related papers  (2022-11-27T11:25:35Z)
• Towards Diverse and Faithful One-shot Adaption of Generative Adversarial Networks [54.80435295622583]
  One-shot generative domain adaption aims to transfer a pre-trained generator on one domain to a new domain using one reference image only. We present a novel one-shot generative domain adaption method, i.e., DiFa, for diverse generation and faithful adaptation.
  arXiv  Detail & Related papers  (2022-07-18T16:29:41Z)
• FewGAN: Generating from the Joint Distribution of a Few Images [95.6635227371479]
  We introduce FewGAN, a generative model for generating novel, high-quality and diverse images. FewGAN is a hierarchical patch-GAN that applies quantization at the first coarse scale, followed by a pyramid of residual fully convolutional GANs at finer scales. In an extensive set of experiments, it is shown that FewGAN outperforms baselines both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2022-07-18T07:11:28Z)
• GIU-GANs: Global Information Utilization for Generative Adversarial Networks [3.3945834638760948]
  In this paper, we propose a new GANs called Involution Generative Adversarial Networks (GIU-GANs) GIU-GANs leverages a brand new module called the Global Information Utilization (GIU) module, which integrates Squeeze-and-Excitation Networks (SENet) and involution. Batch Normalization(BN) inevitably ignores the representation differences among noise sampled by the generator, and thus degrades the generated image quality.
  arXiv  Detail & Related papers  (2022-01-25T17:17:15Z)
• Investigating the Potential of Auxiliary-Classifier GANs for Image Classification in Low Data Regimes [12.128005423388226]
  We examine the potential for Auxiliary-Classifier GANs (AC-GANs) as a 'one-stop-shop' architecture for image classification. AC-GANs show promise in image classification, achieving competitive performance with standard CNNs.
  arXiv  Detail & Related papers  (2022-01-22T19:33:16Z)
• Inducing Optimal Attribute Representations for Conditional GANs [61.24506213440997]
  Conditional GANs are widely used in translating an image from one category to another. Existing conditional GANs commonly encode target domain label information as hard-coded categorical vectors in the form of 0s and 1s. We propose a novel end-to-end learning framework with Graph Convolutional Networks to learn the attribute representations to condition on the generator.
  arXiv  Detail & Related papers  (2020-03-13T20:24:07Z)
• On Leveraging Pretrained GANs for Generation with Limited Data [83.32972353800633]
  generative adversarial networks (GANs) can generate highly realistic images, that are often indistinguishable (by humans) from real images. Most images so generated are not contained in a training dataset, suggesting potential for augmenting training sets with GAN-generated data. We leverage existing GAN models pretrained on large-scale datasets to introduce additional knowledge, following the concept of transfer learning. An extensive set of experiments is presented to demonstrate the effectiveness of the proposed techniques on generation with limited data.
  arXiv  Detail & Related papers  (2020-02-26T21:53:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.