Unsupervised Image-to-Image Translation with Generative Prior

• URL: http://arxiv.org/abs/2204.03641v1
• Date: Thu, 7 Apr 2022 17:59:23 GMT
• Title: Unsupervised Image-to-Image Translation with Generative Prior
• Authors: Shuai Yang, Liming Jiang, Ziwei Liu, Chen Change Loy
• Abstract summary: Unsupervised image-to-image translation aims to learn the translation between two visual domains without paired data. We present a novel framework, Generative Prior-guided UN Image-to-image Translation (GP-UNIT), to improve the overall quality and applicability of the translation algorithm.
• Score: 103.54337984566877
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unsupervised image-to-image translation aims to learn the translation between two visual domains without paired data. Despite the recent progress in image translation models, it remains challenging to build mappings between complex domains with drastic visual discrepancies. In this work, we present a novel framework, Generative Prior-guided UNsupervised Image-to-image Translation (GP-UNIT), to improve the overall quality and applicability of the translation algorithm. Our key insight is to leverage the generative prior from pre-trained class-conditional GANs (e.g., BigGAN) to learn rich content correspondences across various domains. We propose a novel coarse-to-fine scheme: we first distill the generative prior to capture a robust coarse-level content representation that can link objects at an abstract semantic level, based on which fine-level content features are adaptively learned for more accurate multi-level content correspondences. Extensive experiments demonstrate the superiority of our versatile framework over state-of-the-art methods in robust, high-quality and diversified translations, even for challenging and distant domains.

Related papers

• GP-UNIT: Generative Prior for Versatile Unsupervised Image-to-Image Translation [103.54337984566877]
  We introduce a novel versatile framework, Generative Prior-guided UNsupervised Image-to-image Translation (GP-UNIT) GP-UNIT is able to perform valid translations between both close domains and distant domains. We validate the superiority of GP-UNIT over state-of-the-art translation models in robust, high-quality and diversified translations.
  arXiv  Detail & Related papers  (2023-06-07T17:59:22Z)
• Masked and Adaptive Transformer for Exemplar Based Image Translation [16.93344592811513]
  Cross-domain semantic matching is challenging. We propose a masked and adaptive transformer (MAT) for learning accurate cross-domain correspondence. We devise a novel contrastive style learning method, for acquire quality-discriminative style representations.
  arXiv  Detail & Related papers  (2023-03-30T03:21:14Z)
• Pretraining is All You Need for Image-to-Image Translation [59.43151345732397]
  We propose to use pretraining to boost general image-to-image translation. We show that the proposed pretraining-based image-to-image translation (PITI) is capable of synthesizing images of unprecedented realism and faithfulness.
  arXiv  Detail & Related papers  (2022-05-25T17:58:26Z)
• Multi-domain Unsupervised Image-to-Image Translation with Appearance Adaptive Convolution [62.4972011636884]
  We propose a novel multi-domain unsupervised image-to-image translation (MDUIT) framework. We exploit the decomposed content feature and appearance adaptive convolution to translate an image into a target appearance. We show that the proposed method produces visually diverse and plausible results in multiple domains compared to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-02-06T14:12:34Z)
• Leveraging in-domain supervision for unsupervised image-to-image translation tasks via multi-stream generators [4.726777092009554]
  We introduce two techniques to incorporate this invaluable in-domain prior knowledge for the benefit of translation quality. We propose splitting the input data according to semantic masks, explicitly guiding the network to different behavior for the different regions of the image. In addition, we propose training a semantic segmentation network along with the translation task, and to leverage this output as a loss term that improves robustness.
  arXiv  Detail & Related papers  (2021-12-30T15:29:36Z)
• Separating Content and Style for Unsupervised Image-to-Image Translation [20.44733685446886]
  Unsupervised image-to-image translation aims to learn the mapping between two visual domains with unpaired samples. We propose to separate the content code and style code simultaneously in a unified framework. Based on the correlation between the latent features and the high-level domain-invariant tasks, the proposed framework demonstrates superior performance.
  arXiv  Detail & Related papers  (2021-10-27T12:56:50Z)
• Harnessing the Conditioning Sensorium for Improved Image Translation [2.9631016562930546]
  Multi-modal domain translation typically refers to a novel image that inherits certain localized attributes from a 'content' image. We propose a new approach to learn disentangled 'content' and'style' representations from scratch. We define 'content' based on conditioning information extracted by off-the-shelf pre-trained models. We then train our style extractor and image decoder with an easy to optimize set of reconstruction objectives.
  arXiv  Detail & Related papers  (2021-10-13T02:07:43Z)
• Smoothing the Disentangled Latent Style Space for Unsupervised Image-to-Image Translation [56.55178339375146]
  Image-to-Image (I2I) multi-domain translation models are usually evaluated also using the quality of their semantic results. We propose a new training protocol based on three specific losses which help a translation network to learn a smooth and disentangled latent style space.
  arXiv  Detail & Related papers  (2021-06-16T17:58:21Z)
• BalaGAN: Image Translation Between Imbalanced Domains via Cross-Modal Transfer [53.79505340315916]
  We introduce BalaGAN, specifically designed to tackle the domain imbalance problem. We leverage the latent modalities of the richer domain to turn the image-to-image translation problem into a balanced, multi-class, and conditional translation problem. We show that BalaGAN outperforms strong baselines of both unconditioned and style-transfer-based image-to-image translation methods.
  arXiv  Detail & Related papers  (2020-10-05T14:16:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.