Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2 Network

• URL: http://arxiv.org/abs/2010.05713v2
• Date: Tue, 27 Oct 2020 01:18:01 GMT
• Title: Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2 Network
• Authors: Jialu Huang, Jing Liao, Sam Kwong
• Abstract summary: We propose a new I2I translation method that generates a new model in the target domain via a series of model transformations. By feeding the latent vector into the generated model, we can perform I2I translation between the source domain and target domain.
• Score: 73.5062435623908
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image-to-Image (I2I) translation is a heated topic in academia, and it also has been applied in real-world industry for tasks like image synthesis, super-resolution, and colorization. However, traditional I2I translation methods train data in two or more domains together. This requires lots of computation resources. Moreover, the results are of lower quality, and they contain many more artifacts. The training process could be unstable when the data in different domains are not balanced, and modal collapse is more likely to happen. We proposed a new I2I translation method that generates a new model in the target domain via a series of model transformations on a pre-trained StyleGAN2 model in the source domain. After that, we proposed an inversion method to achieve the conversion between an image and its latent vector. By feeding the latent vector into the generated model, we can perform I2I translation between the source domain and target domain. Both qualitative and quantitative evaluations were conducted to prove that the proposed method can achieve outstanding performance in terms of image quality, diversity and semantic similarity to the input and reference images compared to state-of-the-art works.

Related papers

• UVCGAN v2: An Improved Cycle-Consistent GAN for Unpaired Image-to-Image Translation [10.689788782893096]
  An unpaired image-to-image (I2I) translation technique seeks to find a mapping between two domains of data in a fully unsupervised manner. DMs hold the state-of-the-art status on the I2I translation benchmarks in terms of Frechet distance (FID) This work improves a recent UVCGAN model and equips it with modern advancements in model architectures and training procedures.
  arXiv  Detail & Related papers  (2023-03-28T19:46:34Z)
• Smooth image-to-image translations with latent space interpolations [64.8170758294427]
  Multi-domain image-to-image (I2I) translations can transform a source image according to the style of a target domain. We show that our regularization techniques can improve the state-of-the-art I2I translations by a large margin.
  arXiv  Detail & Related papers  (2022-10-03T11:57:30Z)
• BBDM: Image-to-image Translation with Brownian Bridge Diffusion Models [50.39417112077254]
  A novel image-to-image translation method based on the Brownian Bridge Diffusion Model (BBDM) is proposed. To the best of our knowledge, it is the first work that proposes Brownian Bridge diffusion process for image-to-image translation.
  arXiv  Detail & Related papers  (2022-05-16T13:47:02Z)
• Multi-domain Unsupervised Image-to-Image Translation with Appearance Adaptive Convolution [62.4972011636884]
  We propose a novel multi-domain unsupervised image-to-image translation (MDUIT) framework. We exploit the decomposed content feature and appearance adaptive convolution to translate an image into a target appearance. We show that the proposed method produces visually diverse and plausible results in multiple domains compared to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-02-06T14:12:34Z)
• Unaligned Image-to-Image Translation by Learning to Reweight [40.93678165567824]
  Unsupervised image-to-image translation aims at learning the mapping from the source to target domain without using paired images for training. An essential yet restrictive assumption for unsupervised image translation is that the two domains are aligned. We propose to select images based on importance reweighting and develop a method to learn the weights and perform translation simultaneously and automatically.
  arXiv  Detail & Related papers  (2021-09-24T04:08:22Z)
• Fine-Tuning StyleGAN2 For Cartoon Face Generation [0.0]
  We propose a novel image-to-image translation method that generates images of the target domain by finetuning a stylegan2 pretrained model. The stylegan2 model is suitable for unsupervised I2I translation on unbalanced datasets.
  arXiv  Detail & Related papers  (2021-06-22T14:00:10Z)
• Smoothing the Disentangled Latent Style Space for Unsupervised Image-to-Image Translation [56.55178339375146]
  Image-to-Image (I2I) multi-domain translation models are usually evaluated also using the quality of their semantic results. We propose a new training protocol based on three specific losses which help a translation network to learn a smooth and disentangled latent style space.
  arXiv  Detail & Related papers  (2021-06-16T17:58:21Z)
• Unpaired Image-to-Image Translation via Latent Energy Transport [61.62293304236371]
  Image-to-image translation aims to preserve source contents while translating to discriminative target styles between two visual domains. In this paper, we propose to deploy an energy-based model (EBM) in the latent space of a pretrained autoencoder for this task. Our model is the first to be applicable to 1024$times$1024-resolution unpaired image translation.
  arXiv  Detail & Related papers  (2020-12-01T17:18:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.