DomainGallery: Few-shot Domain-driven Image Generation by Attribute-centric Finetuning

• URL: http://arxiv.org/abs/2411.04571v1
• Date: Thu, 07 Nov 2024 09:55:36 GMT
• Title: DomainGallery: Few-shot Domain-driven Image Generation by Attribute-centric Finetuning
• Authors: Yuxuan Duan, Yan Hong, Bo Zhang, Jun Lan, Huijia Zhu, Weiqiang Wang, Jianfu Zhang, Li Niu, Liqing Zhang,
• Abstract summary: DomainGallery is a few-shot domain-driven image generation method. It features prior attribute erasure, attribute disentanglement, regularization and enhancement. Experiments are given to validate the superior performance of DomainGallery on a variety of domain-driven generation scenarios.
• Score: 51.66633704537334
• License:
• Abstract: The recent progress in text-to-image models pretrained on large-scale datasets has enabled us to generate various images as long as we provide a text prompt describing what we want. Nevertheless, the availability of these models is still limited when we expect to generate images that fall into a specific domain either hard to describe or just unseen to the models. In this work, we propose DomainGallery, a few-shot domain-driven image generation method which aims at finetuning pretrained Stable Diffusion on few-shot target datasets in an attribute-centric manner. Specifically, DomainGallery features prior attribute erasure, attribute disentanglement, regularization and enhancement. These techniques are tailored to few-shot domain-driven generation in order to solve key issues that previous works have failed to settle. Extensive experiments are given to validate the superior performance of DomainGallery on a variety of domain-driven generation scenarios. Codes are available at https://github.com/Ldhlwh/DomainGallery.

Related papers

• Domain Bridge: Generative model-based domain forensic for black-box models [20.84645356097581]
  We introduce an enhanced approach to determine not just the general data domain but also its specific attributes. Our approach uses an image embedding model as the encoder and a generative model as the decoder. A key strength of our approach lies in leveraging the expansive dataset, LAION-5B, on which the generative model Stable Diffusion is trained.
  arXiv  Detail & Related papers  (2024-02-07T07:57:43Z)
• Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning [6.288699905490906]
  We propose Subject-Diffusion, a novel open-domain personalized image generation model. Our method outperforms other SOTA frameworks in single, multiple, and human customized image generation.
  arXiv  Detail & Related papers  (2023-07-21T08:09:47Z)
• Towards Diverse and Faithful One-shot Adaption of Generative Adversarial Networks [54.80435295622583]
  One-shot generative domain adaption aims to transfer a pre-trained generator on one domain to a new domain using one reference image only. We present a novel one-shot generative domain adaption method, i.e., DiFa, for diverse generation and faithful adaptation.
  arXiv  Detail & Related papers  (2022-07-18T16:29:41Z)
• Beyond ImageNet Attack: Towards Crafting Adversarial Examples for Black-box Domains [80.11169390071869]
  Adversarial examples have posed a severe threat to deep neural networks due to their transferable nature. We propose a Beyond ImageNet Attack (BIA) to investigate the transferability towards black-box domains. Our methods outperform state-of-the-art approaches by up to 7.71% (towards coarse-grained domains) and 25.91% (towards fine-grained domains) on average.
  arXiv  Detail & Related papers  (2022-01-27T14:04:27Z)
• StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators [63.85888518950824]
  We present a text-driven method that allows shifting a generative model to new domains. We show that through natural language prompts and a few minutes of training, our method can adapt a generator across a multitude of domains.
  arXiv  Detail & Related papers  (2021-08-02T14:46:46Z)
• Few-shot Image Generation with Elastic Weight Consolidation [53.556446614013105]
  Few-shot image generation seeks to generate more data of a given domain, with only few available training examples. We adapt a pretrained model, without introducing any additional parameters, to the few examples of the target domain. We demonstrate the effectiveness of our algorithm by generating high-quality results of different target domains.
  arXiv  Detail & Related papers  (2020-12-04T18:57:13Z)
• TriGAN: Image-to-Image Translation for Multi-Source Domain Adaptation [82.52514546441247]
  We propose the first approach for Multi-Source Domain Adaptation (MSDA) based on Generative Adversarial Networks. Our method is inspired by the observation that the appearance of a given image depends on three factors: the domain, the style and the content. We test our approach using common MSDA benchmarks, showing that it outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-04-19T05:07:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.