Fine-Tuning StyleGAN2 For Cartoon Face Generation

• URL: http://arxiv.org/abs/2106.12445v1
• Date: Tue, 22 Jun 2021 14:00:10 GMT
• Title: Fine-Tuning StyleGAN2 For Cartoon Face Generation
• Authors: Jihye Back
• Abstract summary: We propose a novel image-to-image translation method that generates images of the target domain by finetuning a stylegan2 pretrained model. The stylegan2 model is suitable for unsupervised I2I translation on unbalanced datasets.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent studies have shown remarkable success in the unsupervised image to image (I2I) translation. However, due to the imbalance in the data, learning joint distribution for various domains is still very challenging. Although existing models can generate realistic target images, it's difficult to maintain the structure of the source image. In addition, training a generative model on large data in multiple domains requires a lot of time and computer resources. To address these limitations, we propose a novel image-to-image translation method that generates images of the target domain by finetuning a stylegan2 pretrained model. The stylegan2 model is suitable for unsupervised I2I translation on unbalanced datasets; it is highly stable, produces realistic images, and even learns properly from limited data when applied with simple fine-tuning techniques. Thus, in this paper, we propose new methods to preserve the structure of the source images and generate realistic images in the target domain. The code and results are available at https://github.com/happy-jihye/Cartoon-StyleGan2

Related papers

• Direct Consistency Optimization for Compositional Text-to-Image Personalization [73.94505688626651]
  Text-to-image (T2I) diffusion models, when fine-tuned on a few personal images, are able to generate visuals with a high degree of consistency. We propose to fine-tune the T2I model by maximizing consistency to reference images, while penalizing the deviation from the pretrained model.
  arXiv  Detail & Related papers  (2024-02-19T09:52:41Z)
• UVCGAN v2: An Improved Cycle-Consistent GAN for Unpaired Image-to-Image Translation [10.689788782893096]
  An unpaired image-to-image (I2I) translation technique seeks to find a mapping between two domains of data in a fully unsupervised manner. DMs hold the state-of-the-art status on the I2I translation benchmarks in terms of Frechet distance (FID) This work improves a recent UVCGAN model and equips it with modern advancements in model architectures and training procedures.
  arXiv  Detail & Related papers  (2023-03-28T19:46:34Z)
• InvGAN: Invertible GANs [88.58338626299837]
  InvGAN, short for Invertible GAN, successfully embeds real images to the latent space of a high quality generative model. This allows us to perform image inpainting, merging, and online data augmentation.
  arXiv  Detail & Related papers  (2021-12-08T21:39:00Z)
• toon2real: Translating Cartoon Images to Realistic Images [1.4419517737536707]
  We apply several state-of-the-art models to perform this task; however, they fail to perform good quality translations. We propose a method based on CycleGAN model for image translation from cartoon domain to photo-realistic domain. We demonstrate our experimental results and show that our proposed model has achieved the lowest Frechet Inception Distance score and better results compared to another state-of-the-art technique, UNIT.
  arXiv  Detail & Related papers  (2021-02-01T20:22:05Z)
• Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2 Network [73.5062435623908]
  We propose a new I2I translation method that generates a new model in the target domain via a series of model transformations. By feeding the latent vector into the generated model, we can perform I2I translation between the source domain and target domain.
  arXiv  Detail & Related papers  (2020-10-12T13:51:40Z)
• COCO-FUNIT: Few-Shot Unsupervised Image Translation with a Content Conditioned Style Encoder [70.23358875904891]
  Unsupervised image-to-image translation aims to learn a mapping of an image in a given domain to an analogous image in a different domain. We propose a new few-shot image translation model, COCO-FUNIT, which computes the style embedding of the example images conditioned on the input image. Our model shows effectiveness in addressing the content loss problem.
  arXiv  Detail & Related papers  (2020-07-15T02:01:14Z)
• TuiGAN: Learning Versatile Image-to-Image Translation with Two Unpaired Images [102.4003329297039]
  An unsupervised image-to-image translation (UI2I) task deals with learning a mapping between two domains without paired images. We propose TuiGAN, a generative model that is trained on only two unpaired images and amounts to one-shot unsupervised learning.
  arXiv  Detail & Related papers  (2020-04-09T16:23:59Z)
• Semi-supervised Learning for Few-shot Image-to-Image Translation [89.48165936436183]
  We propose a semi-supervised method for few-shot image translation, called SEMIT. Our method achieves excellent results on four different datasets using as little as 10% of the source labels.
  arXiv  Detail & Related papers  (2020-03-30T22:46:49Z)
• GANILLA: Generative Adversarial Networks for Image to Illustration Translation [12.55972766570669]
  We show that although the current state-of-the-art image-to-image translation models successfully transfer either the style or the content, they fail to transfer both at the same time. We propose a new generator network to address this issue and show that the resulting network strikes a better balance between style and content.
  arXiv  Detail & Related papers  (2020-02-13T17:12:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.