Dual Projection Generative Adversarial Networks for Conditional Image Generation

• URL: http://arxiv.org/abs/2108.09016v1
• Date: Fri, 20 Aug 2021 06:10:38 GMT
• Title: Dual Projection Generative Adversarial Networks for Conditional Image Generation
• Authors: Ligong Han, Martin Renqiang Min, Anastasis Stathopoulos, Yu Tian, Ruijiang Gao, Asim Kadav, Dimitris Metaxas
• Abstract summary: We propose a Dual Projection GAN (P2GAN) model that learns to balance between em data matching and em label matching. We then propose an improved cGAN model with Auxiliary Classification that directly aligns the fake and real conditionals $P(textclass|textimage)$ by minimizing their $f$-divergence.
• Score: 26.563829113916942
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Conditional Generative Adversarial Networks (cGANs) extend the standard unconditional GAN framework to learning joint data-label distributions from samples, and have been established as powerful generative models capable of generating high-fidelity imagery. A challenge of training such a model lies in properly infusing class information into its generator and discriminator. For the discriminator, class conditioning can be achieved by either (1) directly incorporating labels as input or (2) involving labels in an auxiliary classification loss. In this paper, we show that the former directly aligns the class-conditioned fake-and-real data distributions $P(\text{image}|\text{class})$ ({\em data matching}), while the latter aligns data-conditioned class distributions $P(\text{class}|\text{image})$ ({\em label matching}). Although class separability does not directly translate to sample quality and becomes a burden if classification itself is intrinsically difficult, the discriminator cannot provide useful guidance for the generator if features of distinct classes are mapped to the same point and thus become inseparable. Motivated by this intuition, we propose a Dual Projection GAN (P2GAN) model that learns to balance between {\em data matching} and {\em label matching}. We then propose an improved cGAN model with Auxiliary Classification that directly aligns the fake and real conditionals $P(\text{class}|\text{image})$ by minimizing their $f$-divergence. Experiments on a synthetic Mixture of Gaussian (MoG) dataset and a variety of real-world datasets including CIFAR100, ImageNet, and VGGFace2 demonstrate the efficacy of our proposed models.

Related papers

• Exploring Beyond Logits: Hierarchical Dynamic Labeling Based on Embeddings for Semi-Supervised Classification [49.09505771145326]
  We propose a Hierarchical Dynamic Labeling (HDL) algorithm that does not depend on model predictions and utilizes image embeddings to generate sample labels. Our approach has the potential to change the paradigm of pseudo-label generation in semi-supervised learning.
  arXiv  Detail & Related papers  (2024-04-26T06:00:27Z)
• Ref-Diff: Zero-shot Referring Image Segmentation with Generative Models [68.73086826874733]
  We introduce a novel Referring Diffusional segmentor (Ref-Diff) for referring image segmentation. We demonstrate that without a proposal generator, a generative model alone can achieve comparable performance to existing SOTA weakly-supervised models. This indicates that generative models are also beneficial for this task and can complement discriminative models for better referring segmentation.
  arXiv  Detail & Related papers  (2023-08-31T14:55:30Z)
• Score-based Conditional Generation with Fewer Labeled Data by Self-calibrating Classifier Guidance [16.025002076222002]
  Score-based generative models (SGMs) are a popular family of deep generative models that achieve leading image generation quality. We argue that the problem is rooted in the classifier's tendency to overfit without coordinating with the underlying unconditional distribution. We propose improving classifier-guided SGMs by letting the classifier regularize itself.
  arXiv  Detail & Related papers  (2023-07-09T01:41:22Z)
• Label-Retrieval-Augmented Diffusion Models for Learning from Noisy Labels [61.97359362447732]
  Learning from noisy labels is an important and long-standing problem in machine learning for real applications. In this paper, we reformulate the label-noise problem from a generative-model perspective. Our model achieves new state-of-the-art (SOTA) results on all the standard real-world benchmark datasets.
  arXiv  Detail & Related papers  (2023-05-31T03:01:36Z)
• EGC: Image Generation and Classification via a Diffusion Energy-Based Model [59.591755258395594]
  This work introduces an energy-based classifier and generator, namely EGC, which can achieve superior performance in both tasks using a single neural network. EGC achieves competitive generation results compared with state-of-the-art approaches on ImageNet-1k, CelebA-HQ and LSUN Church. This work represents the first successful attempt to simultaneously excel in both tasks using a single set of network parameters.
  arXiv  Detail & Related papers  (2023-04-04T17:59:14Z)
• DualFair: Fair Representation Learning at Both Group and Individual Levels via Contrastive Self-supervision [73.80009454050858]
  This work presents a self-supervised model, called DualFair, that can debias sensitive attributes like gender and race from learned representations. Our model jointly optimize for two fairness criteria - group fairness and counterfactual fairness.
  arXiv  Detail & Related papers  (2023-03-15T07:13:54Z)
• A Unified Generative Adversarial Network Training via Self-Labeling and Self-Attention [38.31735499785227]
  We propose a novel GAN training scheme that can handle any level of labeling in a unified manner. Our scheme introduces a form of artificial labeling that can incorporate manually defined labels, when available. We evaluate our approach on CIFAR-10, STL-10 and SVHN, and show that both self-labeling and self-attention consistently improve the quality of generated data.
  arXiv  Detail & Related papers  (2021-06-18T04:40:26Z)
• Self-supervised GANs with Label Augmentation [43.78253518292111]
  We propose a novel self-supervised GANs framework with label augmentation, i.e., augmenting the GAN labels (real or fake) with the self-supervised pseudo-labels. We demonstrate that the proposed method significantly outperforms competitive baselines on both generative modeling and representation learning.
  arXiv  Detail & Related papers  (2021-06-16T07:58:00Z)
• Learning and Evaluating Representations for Deep One-class Classification [59.095144932794646]
  We present a two-stage framework for deep one-class classification. We first learn self-supervised representations from one-class data, and then build one-class classifiers on learned representations. In experiments, we demonstrate state-of-the-art performance on visual domain one-class classification benchmarks.
  arXiv  Detail & Related papers  (2020-11-04T23:33:41Z)
• Classify and Generate: Using Classification Latent Space Representations for Image Generations [17.184760662429834]
  We propose a discriminative modeling framework that employs manipulated supervised latent representations to reconstruct and generate new samples belonging to a given class. ReGene has higher classification accuracy than existing conditional generative models while being competitive in terms of FID.
  arXiv  Detail & Related papers  (2020-04-16T09:13:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.