Contrastive Learning for Unsupervised Image-to-Image Translation

• URL: http://arxiv.org/abs/2105.03117v1
• Date: Fri, 7 May 2021 08:43:38 GMT
• Title: Contrastive Learning for Unsupervised Image-to-Image Translation
• Authors: Hanbit Lee, Jinseok Seol, Sang-goo Lee
• Abstract summary: We propose an unsupervised image-to-image translation method based on contrastive learning. We randomly sample a pair of images and train the generator to change the appearance of one towards another while keeping the original structure. Experimental results show that our method outperforms the leading unsupervised baselines in terms of visual quality and translation accuracy.
• Score: 10.091669091440396
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image-to-image translation aims to learn a mapping between different groups of visually distinguishable images. While recent methods have shown impressive ability to change even intricate appearance of images, they still rely on domain labels in training a model to distinguish between distinct visual features. Such dependency on labels often significantly limits the scope of applications since consistent and high-quality labels are expensive. Instead, we wish to capture visual features from images themselves and apply them to enable realistic translation without human-generated labels. To this end, we propose an unsupervised image-to-image translation method based on contrastive learning. The key idea is to learn a discriminator that differentiates between distinctive styles and let the discriminator supervise a generator to transfer those styles across images. During training, we randomly sample a pair of images and train the generator to change the appearance of one towards another while keeping the original structure. Experimental results show that our method outperforms the leading unsupervised baselines in terms of visual quality and translation accuracy.

Related papers

• Distractors-Immune Representation Learning with Cross-modal Contrastive Regularization for Change Captioning [71.14084801851381]
  Change captioning aims to succinctly describe the semantic change between a pair of similar images. Most existing methods directly capture the difference between them, which risk obtaining error-prone difference features. We propose a distractors-immune representation learning network that correlates the corresponding channels of two image representations.
  arXiv  Detail & Related papers  (2024-07-16T13:00:33Z)
• A Style-aware Discriminator for Controllable Image Translation [10.338078700632423]
  Current image-to-image translations do not control the output domain beyond the classes used during training. We propose a style-aware discriminator that acts as a critic as well as a style to provide conditions. Experiments on multiple datasets verify that the proposed model outperforms current state-of-the-art image-to-image translation methods.
  arXiv  Detail & Related papers  (2022-03-29T09:13:33Z)
• Spatially Multi-conditional Image Generation [80.04130168156792]
  We propose a novel neural architecture to address the problem of multi-conditional image generation. The proposed method uses a transformer-like architecture operating pixel-wise, which receives the available labels as input tokens. Our experiments on three benchmark datasets demonstrate the clear superiority of our method over the state-of-the-art and the compared baselines.
  arXiv  Detail & Related papers  (2022-03-25T17:57:13Z)
• Separating Content and Style for Unsupervised Image-to-Image Translation [20.44733685446886]
  Unsupervised image-to-image translation aims to learn the mapping between two visual domains with unpaired samples. We propose to separate the content code and style code simultaneously in a unified framework. Based on the correlation between the latent features and the high-level domain-invariant tasks, the proposed framework demonstrates superior performance.
  arXiv  Detail & Related papers  (2021-10-27T12:56:50Z)
• Multi-Label Image Classification with Contrastive Learning [57.47567461616912]
  We show that a direct application of contrastive learning can hardly improve in multi-label cases. We propose a novel framework for multi-label classification with contrastive learning in a fully supervised setting.
  arXiv  Detail & Related papers  (2021-07-24T15:00:47Z)
• Mixed Supervision Learning for Whole Slide Image Classification [88.31842052998319]
  We propose a mixed supervision learning framework for super high-resolution images. During the patch training stage, this framework can make use of coarse image-level labels to refine self-supervised learning. A comprehensive strategy is proposed to suppress pixel-level false positives and false negatives.
  arXiv  Detail & Related papers  (2021-07-02T09:46:06Z)
• Semantic Diversity Learning for Zero-Shot Multi-label Classification [14.480713752871523]
  This study introduces an end-to-end model training for multi-label zero-shot learning. We propose to use an embedding matrix having principal embedding vectors trained using a tailored loss function. In addition, during training, we suggest up-weighting in the loss function image samples presenting higher semantic diversity to encourage the diversity of the embedding matrix.
  arXiv  Detail & Related papers  (2021-05-12T19:39:07Z)
• Unsupervised Deep Metric Learning with Transformed Attention Consistency and Contrastive Clustering Loss [28.17607283348278]
  Existing approaches for unsupervised metric learning focus on exploring self-supervision information within the input image itself. We observe that, when analyzing images, human eyes often compare images against each other instead of examining images individually. We develop a new approach to unsupervised deep metric learning where the network is learned based on self-supervision information across images.
  arXiv  Detail & Related papers  (2020-08-10T19:33:47Z)
• Contrastive Learning for Unpaired Image-to-Image Translation [64.47477071705866]
  In image-to-image translation, each patch in the output should reflect the content of the corresponding patch in the input, independent of domain. We propose a framework based on contrastive learning to maximize mutual information between the two. We demonstrate that our framework enables one-sided translation in the unpaired image-to-image translation setting, while improving quality and reducing training time.
  arXiv  Detail & Related papers  (2020-07-30T17:59:58Z)
• Distilling Localization for Self-Supervised Representation Learning [82.79808902674282]
  Contrastive learning has revolutionized unsupervised representation learning. Current contrastive models are ineffective at localizing the foreground object. We propose a data-driven approach for learning in variance to backgrounds.
  arXiv  Detail & Related papers  (2020-04-14T16:29:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.