StyleDomain: Efficient and Lightweight Parameterizations of StyleGAN for One-shot and Few-shot Domain Adaptation

• URL: http://arxiv.org/abs/2212.10229v4
• Date: Tue, 12 Sep 2023 09:23:39 GMT
• Title: StyleDomain: Efficient and Lightweight Parameterizations of StyleGAN for One-shot and Few-shot Domain Adaptation
• Authors: Aibek Alanov, Vadim Titov, Maksim Nakhodnov, Dmitry Vetrov
• Abstract summary: We provide a systematic and in-depth analysis of the domain adaptation problem of GANs, focusing on the StyleGAN model. We propose new efficient and lightweight parameterizations of StyleGAN for domain adaptation.
• Score: 4.943054375935879
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Domain adaptation of GANs is a problem of fine-tuning GAN models pretrained on a large dataset (e.g. StyleGAN) to a specific domain with few samples (e.g. painting faces, sketches, etc.). While there are many methods that tackle this problem in different ways, there are still many important questions that remain unanswered. In this paper, we provide a systematic and in-depth analysis of the domain adaptation problem of GANs, focusing on the StyleGAN model. We perform a detailed exploration of the most important parts of StyleGAN that are responsible for adapting the generator to a new domain depending on the similarity between the source and target domains. As a result of this study, we propose new efficient and lightweight parameterizations of StyleGAN for domain adaptation. Particularly, we show that there exist directions in StyleSpace (StyleDomain directions) that are sufficient for adapting to similar domains. For dissimilar domains, we propose Affine+ and AffineLight+ parameterizations that allows us to outperform existing baselines in few-shot adaptation while having significantly less training parameters. Finally, we examine StyleDomain directions and discover their many surprising properties that we apply for domain mixing and cross-domain image morphing. Source code can be found at https://github.com/AIRI-Institute/StyleDomain.

Related papers

• Domain-Rectifying Adapter for Cross-Domain Few-Shot Segmentation [40.667166043101076]
  We propose a small adapter for rectifying diverse target domain styles to the source domain. The adapter is trained to rectify the image features from diverse synthesized target domains to align with the source domain. Our method achieves promising results on cross-domain few-shot semantic segmentation tasks.
  arXiv  Detail & Related papers  (2024-04-16T07:07:40Z)
• HyperDomainNet: Universal Domain Adaptation for Generative Adversarial Networks [0.0]
  We introduce a novel domain-modulation technique that allows to optimize only 6 thousand-dimensional vector instead of 30 million weights of StyleGAN2 to adapt to a target domain. Inspired by the reduction in the size of the optimizing parameter space we consider the problem of multi-domain adaptation of GANs. We propose the HyperDomainNet that is a hypernetwork that predicts our parameterization given the target domain.
  arXiv  Detail & Related papers  (2022-10-17T09:27:39Z)
• Continual Unsupervised Domain Adaptation for Semantic Segmentation using a Class-Specific Transfer [9.46677024179954]
  segmentation models do not generalize to unseen domains. We propose a light-weight style transfer framework that incorporates two class-conditional AdaIN layers. We extensively validate our approach on a synthetic sequence and further propose a challenging sequence consisting of real domains.
  arXiv  Detail & Related papers  (2022-08-12T21:30:49Z)
• Dynamic Instance Domain Adaptation [109.53575039217094]
  Most studies on unsupervised domain adaptation assume that each domain's training samples come with domain labels. We develop a dynamic neural network with adaptive convolutional kernels to generate instance-adaptive residuals to adapt domain-agnostic deep features to each individual instance. Our model, dubbed DIDA-Net, achieves state-of-the-art performance on several commonly used single-source and multi-source UDA datasets.
  arXiv  Detail & Related papers  (2022-03-09T20:05:54Z)
• Domain Adaptation via Prompt Learning [39.97105851723885]
  Unsupervised domain adaption (UDA) aims to adapt models learned from a well-annotated source domain to a target domain. We introduce a novel prompt learning paradigm for UDA, named Domain Adaptation via Prompt Learning (DAPL)
  arXiv  Detail & Related papers  (2022-02-14T13:25:46Z)
• Self-Adversarial Disentangling for Specific Domain Adaptation [52.1935168534351]
  Domain adaptation aims to bridge the domain shifts between the source and target domains. Recent methods typically do not consider explicit prior knowledge on a specific dimension.
  arXiv  Detail & Related papers  (2021-08-08T02:36:45Z)
• Dynamic Transfer for Multi-Source Domain Adaptation [82.54405157719641]
  We present dynamic transfer to address domain conflicts, where the model parameters are adapted to samples. It breaks down source domain barriers and turns multi-source domains into a single-source domain. Experimental results show that, without using domain labels, our dynamic transfer outperforms the state-of-the-art method by more than 3%.
  arXiv  Detail & Related papers  (2021-03-19T01:22:12Z)
• Domain Adaptation for Semantic Parsing [68.81787666086554]
  We propose a novel semantic for domain adaptation, where we have much fewer annotated data in the target domain compared to the source domain. Our semantic benefits from a two-stage coarse-to-fine framework, thus can provide different and accurate treatments for the two stages. Experiments on a benchmark dataset show that our method consistently outperforms several popular domain adaptation strategies.
  arXiv  Detail & Related papers  (2020-06-23T14:47:41Z)
• Cross-domain Detection via Graph-induced Prototype Alignment [114.8952035552862]
  We propose a Graph-induced Prototype Alignment (GPA) framework to seek for category-level domain alignment. In addition, in order to alleviate the negative effect of class-imbalance on domain adaptation, we design a Class-reweighted Contrastive Loss. Our approach outperforms existing methods with a remarkable margin.
  arXiv  Detail & Related papers  (2020-03-28T17:46:55Z)
• Differential Treatment for Stuff and Things: A Simple Unsupervised Domain Adaptation Method for Semantic Segmentation [105.96860932833759]
  State-of-the-art approaches prove that performing semantic-level alignment is helpful in tackling the domain shift issue. We propose to improve the semantic-level alignment with different strategies for stuff regions and for things. In addition to our proposed method, we show that our method can help ease this issue by minimizing the most similar stuff and instance features between the source and the target domains.
  arXiv  Detail & Related papers  (2020-03-18T04:43:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.