Dual Contrastive Learning for Unsupervised Image-to-Image Translation

• URL: http://arxiv.org/abs/2104.07689v1
• Date: Thu, 15 Apr 2021 18:00:22 GMT
• Title: Dual Contrastive Learning for Unsupervised Image-to-Image Translation
• Authors: Junlin Han, Mehrdad Shoeiby, Lars Petersson, Mohammad Ali Armin
• Abstract summary: Unsupervised image-to-image translation tasks aim to find a mapping between a source domain X and a target domain Y from unpaired training data. Contrastive learning for Unpaired image-to-image Translation yields state-of-the-art results. We propose a novel method based on contrastive learning and a dual learning setting to infer an efficient mapping between unpaired data.
• Score: 16.759958400617947
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unsupervised image-to-image translation tasks aim to find a mapping between a source domain X and a target domain Y from unpaired training data. Contrastive learning for Unpaired image-to-image Translation (CUT) yields state-of-the-art results in modeling unsupervised image-to-image translation by maximizing mutual information between input and output patches using only one encoder for both domains. In this paper, we propose a novel method based on contrastive learning and a dual learning setting (exploiting two encoders) to infer an efficient mapping between unpaired data. Additionally, while CUT suffers from mode collapse, a variant of our method efficiently addresses this issue. We further demonstrate the advantage of our approach through extensive ablation studies demonstrating superior performance comparing to recent approaches in multiple challenging image translation tasks. Lastly, we demonstrate that the gap between unsupervised methods and supervised methods can be efficiently closed.

Related papers

• Multi-cropping Contrastive Learning and Domain Consistency for Unsupervised Image-to-Image Translation [5.562419999563734]
  We propose a novel unsupervised image-to-image translation framework based on multi-cropping contrastive learning and domain consistency, called MCDUT. In many image-to-image translation tasks, our method achieves state-of-the-art results, and the advantages of our method have been proven through comparison experiments and ablation research.
  arXiv  Detail & Related papers  (2023-04-24T16:20:28Z)
• Unsupervised Multi-Modal Medical Image Registration via Discriminator-Free Image-to-Image Translation [4.43142018105102]
  We propose a novel translation-based unsupervised deformable image registration approach to convert the multi-modal registration problem to a mono-modal one. Our approach incorporates a discriminator-free translation network to facilitate the training of the registration network and a patchwise contrastive loss to encourage the translation network to preserve object shapes.
  arXiv  Detail & Related papers  (2022-04-28T17:18:21Z)
• Multi-domain Unsupervised Image-to-Image Translation with Appearance Adaptive Convolution [62.4972011636884]
  We propose a novel multi-domain unsupervised image-to-image translation (MDUIT) framework. We exploit the decomposed content feature and appearance adaptive convolution to translate an image into a target appearance. We show that the proposed method produces visually diverse and plausible results in multiple domains compared to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-02-06T14:12:34Z)
• StEP: Style-based Encoder Pre-training for Multi-modal Image Synthesis [68.3787368024951]
  We propose a novel approach for multi-modal Image-to-image (I2I) translation. We learn a latent embedding, jointly with the generator, that models the variability of the output domain. Specifically, we pre-train a generic style encoder using a novel proxy task to learn an embedding of images, from arbitrary domains, into a low-dimensional style latent space.
  arXiv  Detail & Related papers  (2021-04-14T19:58:24Z)
• Multiple GAN Inversion for Exemplar-based Image-to-Image Translation [0.0]
  We propose Multiple GAN Inversion for Exemplar-based Image-to-Image Translation. Our novel Multiple GAN Inversion avoids human intervention using a self-deciding algorithm in choosing the number of layers. Experimental results shows the advantage of the proposed method compared to existing state-of-the-art exemplar-based image-to-image translation methods.
  arXiv  Detail & Related papers  (2021-03-26T13:46:14Z)
• GAIT: Gradient Adjusted Unsupervised Image-to-Image Translation [5.076419064097734]
  An adversarial loss is utilized to match the distributions of the translated and target image sets. This may create artifacts if two domains have different marginal distributions, for example, in uniform areas. We propose an unsupervised IIT that preserves the uniform regions after the translation.
  arXiv  Detail & Related papers  (2020-09-02T08:04:00Z)
• Delving into Inter-Image Invariance for Unsupervised Visual Representations [108.33534231219464]
  We present a study to better understand the role of inter-image invariance learning. Online labels converge faster than offline labels. Semi-hard negative samples are more reliable and unbiased than hard negative samples.
  arXiv  Detail & Related papers  (2020-08-26T17:44:23Z)
• Contrastive Learning for Unpaired Image-to-Image Translation [64.47477071705866]
  In image-to-image translation, each patch in the output should reflect the content of the corresponding patch in the input, independent of domain. We propose a framework based on contrastive learning to maximize mutual information between the two. We demonstrate that our framework enables one-sided translation in the unpaired image-to-image translation setting, while improving quality and reducing training time.
  arXiv  Detail & Related papers  (2020-07-30T17:59:58Z)
• COCO-FUNIT: Few-Shot Unsupervised Image Translation with a Content Conditioned Style Encoder [70.23358875904891]
  Unsupervised image-to-image translation aims to learn a mapping of an image in a given domain to an analogous image in a different domain. We propose a new few-shot image translation model, COCO-FUNIT, which computes the style embedding of the example images conditioned on the input image. Our model shows effectiveness in addressing the content loss problem.
  arXiv  Detail & Related papers  (2020-07-15T02:01:14Z)
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments [57.33699905852397]
  We propose an online algorithm, SwAV, that takes advantage of contrastive methods without requiring to compute pairwise comparisons. Our method simultaneously clusters the data while enforcing consistency between cluster assignments. Our method can be trained with large and small batches and can scale to unlimited amounts of data.
  arXiv  Detail & Related papers  (2020-06-17T14:00:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.