ContraGAN: Contrastive Learning for Conditional Image Generation

• URL: http://arxiv.org/abs/2006.12681v3
• Date: Wed, 3 Feb 2021 05:34:11 GMT
• Title: ContraGAN: Contrastive Learning for Conditional Image Generation
• Authors: Minguk Kang and Jaesik Park
• Abstract summary: Conditional Generative Adversarial Networks (GAN) are used to generate diverse images using class label information. We propose ContraGAN that considers relations between multiple image embeddings in the same batch (data-to-data relations) as well as the data-to-class relations by using a conditional contrastive loss. The experimental results show that ContraGAN outperforms state-of-the-art-models by 7.3% and 7.7% on Tiny ImageNet and ImageNet datasets, respectively.
• Score: 14.077997868828177
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conditional image generation is the task of generating diverse images using class label information. Although many conditional Generative Adversarial Networks (GAN) have shown realistic results, such methods consider pairwise relations between the embedding of an image and the embedding of the corresponding label (data-to-class relations) as the conditioning losses. In this paper, we propose ContraGAN that considers relations between multiple image embeddings in the same batch (data-to-data relations) as well as the data-to-class relations by using a conditional contrastive loss. The discriminator of ContraGAN discriminates the authenticity of given samples and minimizes a contrastive objective to learn the relations between training images. Simultaneously, the generator tries to generate realistic images that deceive the authenticity and have a low contrastive loss. The experimental results show that ContraGAN outperforms state-of-the-art-models by 7.3% and 7.7% on Tiny ImageNet and ImageNet datasets, respectively. Besides, we experimentally demonstrate that contrastive learning helps to relieve the overfitting of the discriminator. For a fair comparison, we re-implement twelve state-of-the-art GANs using the PyTorch library. The software package is available at https://github.com/POSTECH-CVLab/PyTorch-StudioGAN.

Related papers

• Fine-grained Image-to-LiDAR Contrastive Distillation with Visual Foundation Models [55.99654128127689]
  Visual Foundation Models (VFMs) are used to enhance 3D representation learning. VFMs generate semantic labels for weakly-supervised pixel-to-point contrastive distillation. We adapt sampling probabilities of points to address imbalances in spatial distribution and category frequency.
  arXiv  Detail & Related papers  (2024-05-23T07:48:19Z)
• Image Similarity using An Ensemble of Context-Sensitive Models [2.9490616593440317]
  We present a more intuitive approach to build and compare image similarity models based on labelled data. We address the challenges of sparse sampling in the image space (R, A, B) and biases in the models trained with context-based data. Our testing results show that the ensemble model constructed performs 5% better than the best individual context-sensitive models.
  arXiv  Detail & Related papers  (2024-01-15T20:23:05Z)
• Similarity Contrastive Estimation for Image and Video Soft Contrastive Self-Supervised Learning [0.22940141855172028]
  We propose a novel formulation of contrastive learning using semantic similarity between instances. Our training objective is a soft contrastive one that brings the positives closer and estimates a continuous distribution to push or pull negative instances. We show that SCE reaches state-of-the-art results for pretraining video representation and that the learned representation can generalize to video downstream tasks.
  arXiv  Detail & Related papers  (2022-12-21T16:56:55Z)
• What can we learn about a generated image corrupting its latent representation? [57.1841740328509]
  We investigate the hypothesis that we can predict image quality based on its latent representation in the GANs bottleneck. We achieve this by corrupting the latent representation with noise and generating multiple outputs.
  arXiv  Detail & Related papers  (2022-10-12T14:40:32Z)
• Exploring Negatives in Contrastive Learning for Unpaired Image-to-Image Translation [12.754320302262533]
  We introduce a new negative Pruning technology for Unpaired image-to-image Translation (PUT) by sparsifying and ranking the patches. The proposed algorithm is efficient, flexible and enables the model to learn essential information between corresponding patches stably.
  arXiv  Detail & Related papers  (2022-04-23T08:31:18Z)
• Modulated Contrast for Versatile Image Synthesis [60.304183493234376]
  MoNCE is a versatile metric that introduces image contrast to learn a calibrated metric for the perception of multifaceted inter-image distances. We introduce optimal transport in MoNCE to modulate the pushing force of negative samples collaboratively across multiple contrastive objectives.
  arXiv  Detail & Related papers  (2022-03-17T14:03:46Z)
• Robust Contrastive Learning against Noisy Views [79.71880076439297]
  We propose a new contrastive loss function that is robust against noisy views. We show that our approach provides consistent improvements over the state-of-the-art image, video, and graph contrastive learning benchmarks.
  arXiv  Detail & Related papers  (2022-01-12T05:24:29Z)
• DivCo: Diverse Conditional Image Synthesis via Contrastive Generative Adversarial Network [70.12848483302915]
  Conditional generative adversarial networks (cGANs) target at diverse images given the input conditions and latent codes. Recent MSGAN tried to encourage the diversity of the generated image but only considers "negative" relations between the image pairs. We propose a novel DivCo framework to properly constrain both "positive" and "negative" relations between the generated images specified in the latent space.
  arXiv  Detail & Related papers  (2021-03-14T11:11:15Z)
• Delving into Inter-Image Invariance for Unsupervised Visual Representations [108.33534231219464]
  We present a study to better understand the role of inter-image invariance learning. Online labels converge faster than offline labels. Semi-hard negative samples are more reliable and unbiased than hard negative samples.
  arXiv  Detail & Related papers  (2020-08-26T17:44:23Z)
• A Self-supervised GAN for Unsupervised Few-shot Object Recognition [39.79912546252623]
  This paper addresses unsupervised few-shot object recognition. All training images are unlabeled, and test images are divided into queries and a few labeled support images per object class of interest. We extend the vanilla GAN with two loss functions, both aimed at self-supervised learning.
  arXiv  Detail & Related papers  (2020-08-16T19:47:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.