Return of Unconditional Generation: A Self-supervised Representation Generation Method

• URL: http://arxiv.org/abs/2312.03701v4
• Date: Fri, 01 Nov 2024 14:48:57 GMT
• Title: Return of Unconditional Generation: A Self-supervised Representation Generation Method
• Authors: Tianhong Li, Dina Katabi, Kaiming He,
• Abstract summary: Unconditional generation is a problem of modeling data distribution without relying on human-annotated labels. In this work, we show that one can close this gap by generating semantic representations in the representation space produced by a self-supervised encoder. This framework, called Representation-Conditioned Generation (RCG), provides an effective solution to the unconditional generation problem without using labels.
• Score: 36.27605000082541
• License:
• Abstract: Unconditional generation -- the problem of modeling data distribution without relying on human-annotated labels -- is a long-standing and fundamental challenge in generative models, creating a potential of learning from large-scale unlabeled data. In the literature, the generation quality of an unconditional method has been much worse than that of its conditional counterpart. This gap can be attributed to the lack of semantic information provided by labels. In this work, we show that one can close this gap by generating semantic representations in the representation space produced by a self-supervised encoder. These representations can be used to condition the image generator. This framework, called Representation-Conditioned Generation (RCG), provides an effective solution to the unconditional generation problem without using labels. Through comprehensive experiments, we observe that RCG significantly improves unconditional generation quality: e.g., it achieves a new state-of-the-art FID of 2.15 on ImageNet 256x256, largely reducing the previous best of 5.91 by a relative 64%. Our unconditional results are situated in the same tier as the leading class-conditional ones. We hope these encouraging observations will attract the community's attention to the fundamental problem of unconditional generation. Code is available at https://github.com/LTH14/rcg.

Related papers

• Tractable Transformers for Flexible Conditional Generation [44.52426555357705]
  Tractable Transformers (Tracformer) is a Transformer-based generative model that is more robust to different conditional generation tasks. This paper proposes Tractable Transformers, a Transformer-based generative model that is more robust to different conditional generation tasks.
  arXiv  Detail & Related papers  (2025-02-11T15:05:26Z)
• Score-based Conditional Generation with Fewer Labeled Data by Self-calibrating Classifier Guidance [16.025002076222002]
  Score-based generative models (SGMs) are a popular family of deep generative models that achieve leading image generation quality. We argue that the problem is rooted in the classifier's tendency to overfit without coordinating with the underlying unconditional distribution. We propose improving classifier-guided SGMs by letting the classifier regularize itself.
  arXiv  Detail & Related papers  (2023-07-09T01:41:22Z)
• Generator Knows What Discriminator Should Learn in Unconditional GANs [18.913330654689496]
  We propose a new generator-guided discriminator regularization(GGDR) in which the generator feature maps supervise the discriminator to have rich semantic representations in unconditional generation. In specific, we employ an U-Net architecture for discriminator, which is trained to predict the generator feature maps given fake images as inputs.
  arXiv  Detail & Related papers  (2022-07-27T06:49:26Z)
• Spatially Multi-conditional Image Generation [80.04130168156792]
  We propose a novel neural architecture to address the problem of multi-conditional image generation. The proposed method uses a transformer-like architecture operating pixel-wise, which receives the available labels as input tokens. Our experiments on three benchmark datasets demonstrate the clear superiority of our method over the state-of-the-art and the compared baselines.
  arXiv  Detail & Related papers  (2022-03-25T17:57:13Z)
• Self-Conditioned Generative Adversarial Networks for Image Editing [61.50205580051405]
  Generative Adversarial Networks (GANs) are susceptible to bias, learned from either the unbalanced data, or through mode collapse. We argue that this bias is responsible not only for fairness concerns, but that it plays a key role in the collapse of latent-traversal editing methods when deviating away from the distribution's core.
  arXiv  Detail & Related papers  (2022-02-08T18:08:24Z)
• Collapse by Conditioning: Training Class-conditional GANs with Limited Data [109.30895503994687]
  We propose a training strategy for conditional GANs (cGANs) that effectively prevents the observed mode-collapse by leveraging unconditional learning. Our training strategy starts with an unconditional GAN and gradually injects conditional information into the generator and the objective function. The proposed method for training cGANs with limited data results not only in stable training but also in generating high-quality images.
  arXiv  Detail & Related papers  (2022-01-17T18:59:23Z)
• Dual Projection Generative Adversarial Networks for Conditional Image Generation [26.563829113916942]
  We propose a Dual Projection GAN (P2GAN) model that learns to balance between em data matching and em label matching. We then propose an improved cGAN model with Auxiliary Classification that directly aligns the fake and real conditionals $P(textclass|textimage)$ by minimizing their $f$-divergence.
  arXiv  Detail & Related papers  (2021-08-20T06:10:38Z)
• Guiding GANs: How to control non-conditional pre-trained GANs for conditional image generation [69.10717733870575]
  We present a novel method for guiding generic non-conditional GANs to behave as conditional GANs. Our approach adds into the mix an encoder network to generate the high-dimensional random input that are fed to the generator network of a non-conditional GAN.
  arXiv  Detail & Related papers  (2021-01-04T14:03:32Z)
• Inducing Optimal Attribute Representations for Conditional GANs [61.24506213440997]
  Conditional GANs are widely used in translating an image from one category to another. Existing conditional GANs commonly encode target domain label information as hard-coded categorical vectors in the form of 0s and 1s. We propose a novel end-to-end learning framework with Graph Convolutional Networks to learn the attribute representations to condition on the generator.
  arXiv  Detail & Related papers  (2020-03-13T20:24:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.