TuiGAN: Learning Versatile Image-to-Image Translation with Two Unpaired
Images
- URL: http://arxiv.org/abs/2004.04634v2
- Date: Sun, 9 Aug 2020 02:23:03 GMT
- Title: TuiGAN: Learning Versatile Image-to-Image Translation with Two Unpaired
Images
- Authors: Jianxin Lin, Yingxue Pang, Yingce Xia, Zhibo Chen, Jiebo Luo
- Abstract summary: An unsupervised image-to-image translation (UI2I) task deals with learning a mapping between two domains without paired images.
We propose TuiGAN, a generative model that is trained on only two unpaired images and amounts to one-shot unsupervised learning.
- Score: 102.4003329297039
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: An unsupervised image-to-image translation (UI2I) task deals with learning a
mapping between two domains without paired images. While existing UI2I methods
usually require numerous unpaired images from different domains for training,
there are many scenarios where training data is quite limited. In this paper,
we argue that even if each domain contains a single image, UI2I can still be
achieved. To this end, we propose TuiGAN, a generative model that is trained on
only two unpaired images and amounts to one-shot unsupervised learning. With
TuiGAN, an image is translated in a coarse-to-fine manner where the generated
image is gradually refined from global structures to local details. We conduct
extensive experiments to verify that our versatile method can outperform strong
baselines on a wide variety of UI2I tasks. Moreover, TuiGAN is capable of
achieving comparable performance with the state-of-the-art UI2I models trained
with sufficient data.
Related papers
- Enhance Image Classification via Inter-Class Image Mixup with Diffusion Model [80.61157097223058]
A prevalent strategy to bolster image classification performance is through augmenting the training set with synthetic images generated by T2I models.
In this study, we scrutinize the shortcomings of both current generative and conventional data augmentation techniques.
We introduce an innovative inter-class data augmentation method known as Diff-Mix, which enriches the dataset by performing image translations between classes.
arXiv Detail & Related papers (2024-03-28T17:23:45Z) - Improving Human-Object Interaction Detection via Virtual Image Learning [68.56682347374422]
Human-Object Interaction (HOI) detection aims to understand the interactions between humans and objects.
In this paper, we propose to alleviate the impact of such an unbalanced distribution via Virtual Image Leaning (VIL)
A novel label-to-image approach, Multiple Steps Image Creation (MUSIC), is proposed to create a high-quality dataset that has a consistent distribution with real images.
arXiv Detail & Related papers (2023-08-04T10:28:48Z) - Synthetic-to-Real Domain Adaptation using Contrastive Unpaired
Translation [28.19031441659854]
We propose a multi-step method to obtain training data without manual annotation effort.
From 3D object meshes, we generate images using a modern synthesis pipeline.
We utilize a state-of-the-art image-to-image translation method to adapt the synthetic images to the real domain.
arXiv Detail & Related papers (2022-03-17T17:13:23Z) - Multi-domain Unsupervised Image-to-Image Translation with Appearance
Adaptive Convolution [62.4972011636884]
We propose a novel multi-domain unsupervised image-to-image translation (MDUIT) framework.
We exploit the decomposed content feature and appearance adaptive convolution to translate an image into a target appearance.
We show that the proposed method produces visually diverse and plausible results in multiple domains compared to the state-of-the-art methods.
arXiv Detail & Related papers (2022-02-06T14:12:34Z) - Leveraging in-domain supervision for unsupervised image-to-image
translation tasks via multi-stream generators [4.726777092009554]
We introduce two techniques to incorporate this invaluable in-domain prior knowledge for the benefit of translation quality.
We propose splitting the input data according to semantic masks, explicitly guiding the network to different behavior for the different regions of the image.
In addition, we propose training a semantic segmentation network along with the translation task, and to leverage this output as a loss term that improves robustness.
arXiv Detail & Related papers (2021-12-30T15:29:36Z) - Fine-Tuning StyleGAN2 For Cartoon Face Generation [0.0]
We propose a novel image-to-image translation method that generates images of the target domain by finetuning a stylegan2 pretrained model.
The stylegan2 model is suitable for unsupervised I2I translation on unbalanced datasets.
arXiv Detail & Related papers (2021-06-22T14:00:10Z) - Self-Supervised Ranking for Representation Learning [108.38993212650577]
We present a new framework for self-supervised representation learning by formulating it as a ranking problem in an image retrieval context.
We train a representation encoder by maximizing average precision (AP) for ranking, where random views of an image are considered positively related.
In principle, by using a ranking criterion, we eliminate reliance on object-centric curated datasets.
arXiv Detail & Related papers (2020-10-14T17:24:56Z) - Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2
Network [73.5062435623908]
We propose a new I2I translation method that generates a new model in the target domain via a series of model transformations.
By feeding the latent vector into the generated model, we can perform I2I translation between the source domain and target domain.
arXiv Detail & Related papers (2020-10-12T13:51:40Z) - Crossing-Domain Generative Adversarial Networks for Unsupervised
Multi-Domain Image-to-Image Translation [12.692904507625036]
We propose a general framework for unsupervised image-to-image translation across multiple domains.
Our proposed framework consists of a pair of encoders along with a pair of GANs which learns high-level features across different domains to generate diverse and realistic samples from.
arXiv Detail & Related papers (2020-08-27T01:54:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.