MyStyle: A Personalized Generative Prior
- URL: http://arxiv.org/abs/2203.17272v1
- Date: Thu, 31 Mar 2022 17:59:19 GMT
- Title: MyStyle: A Personalized Generative Prior
- Authors: Yotam Nitzan, Kfir Aberman, Qiurui He, Orly Liba, Michal Yarom, Yossi
Gandelsman, Inbar Mosseri, Yael Pritch, Daniel Cohen-or
- Abstract summary: We introduce MyStyle, a personalized deep generative prior trained with a few shots of an individual.
MyStyle allows to reconstruct, enhance and edit images of a specific person.
- Score: 38.3436972491162
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We introduce MyStyle, a personalized deep generative prior trained with a few
shots of an individual. MyStyle allows to reconstruct, enhance and edit images
of a specific person, such that the output is faithful to the person's key
facial characteristics. Given a small reference set of portrait images of a
person (~100), we tune the weights of a pretrained StyleGAN face generator to
form a local, low-dimensional, personalized manifold in the latent space. We
show that this manifold constitutes a personalized region that spans latent
codes associated with diverse portrait images of the individual. Moreover, we
demonstrate that we obtain a personalized generative prior, and propose a
unified approach to apply it to various ill-posed image enhancement problems,
such as inpainting and super-resolution, as well as semantic editing. Using the
personalized generative prior we obtain outputs that exhibit high-fidelity to
the input images and are also faithful to the key facial characteristics of the
individual in the reference set. We demonstrate our method with fair-use images
of numerous widely recognizable individuals for whom we have the prior
knowledge for a qualitative evaluation of the expected outcome. We evaluate our
approach against few-shots baselines and show that our personalized prior,
quantitatively and qualitatively, outperforms state-of-the-art alternatives.
Related papers
- FlashFace: Human Image Personalization with High-fidelity Identity Preservation [59.76645602354481]
FlashFace allows users to easily personalize their own photos by providing one or a few reference face images and a text prompt.
Our approach is distinguishable from existing human photo customization methods by higher-fidelity identity preservation and better instruction following.
arXiv Detail & Related papers (2024-03-25T17:59:57Z) - IDAdapter: Learning Mixed Features for Tuning-Free Personalization of Text-to-Image Models [31.762112403595612]
IDAdapter is a tuning-free approach that enhances the diversity and identity preservation in personalized image generation from a single face image.
During the training phase, we incorporate mixed features from multiple reference images of a specific identity to enrich identity-related content details.
arXiv Detail & Related papers (2024-03-20T12:13:04Z) - Pick-and-Draw: Training-free Semantic Guidance for Text-to-Image
Personalization [56.12990759116612]
Pick-and-Draw is a training-free semantic guidance approach to boost identity consistency and generative diversity for personalization methods.
The proposed approach can be applied to any personalized diffusion models and requires as few as a single reference image.
arXiv Detail & Related papers (2024-01-30T05:56:12Z) - PALP: Prompt Aligned Personalization of Text-to-Image Models [68.91005384187348]
Existing personalization methods compromise personalization ability or the alignment to complex prompts.
We propose a new approach focusing on personalization methods for a emphsingle prompt to address this issue.
Our method excels in improving text alignment, enabling the creation of images with complex and intricate prompts.
arXiv Detail & Related papers (2024-01-11T18:35:33Z) - Personalized Restoration via Dual-Pivot Tuning [18.912158172904654]
We propose a simple, yet effective, method for personalized restoration, called Dual-Pivot Tuning.
Our key observation is that for optimal personalization, the generative model should be tuned around a fixed text pivot.
This approach ensures that personalization does not interfere with the restoration process, resulting in a natural appearance with high fidelity to the person's identity and the attributes of the degraded image.
arXiv Detail & Related papers (2023-12-28T18:57:49Z) - MyStyle++: A Controllable Personalized Generative Prior [13.030741541265428]
We build upon MyStyle, a recently introduced method, that tunes the weights of a pre-trained StyleGAN face generator on a few images of an individual.
MyStyle does not demonstrate precise control over the attributes of the generated images.
We demonstrate that our approach, dubbed MyStyle++, is able to synthesize, edit, and enhance images of an individual with great control over the attributes.
arXiv Detail & Related papers (2023-06-08T01:35:43Z) - Identity Encoder for Personalized Diffusion [57.1198884486401]
We propose an encoder-based approach for personalization.
We learn an identity encoder which can extract an identity representation from a set of reference images of a subject.
We show that our approach consistently outperforms existing fine-tuning based approach in both image generation and reconstruction.
arXiv Detail & Related papers (2023-04-14T23:32:24Z) - Generating Person Images with Appearance-aware Pose Stylizer [66.44220388377596]
We present a novel end-to-end framework to generate realistic person images based on given person poses and appearances.
The core of our framework is a novel generator called Appearance-aware Pose Stylizer (APS) which generates human images by coupling the target pose with the conditioned person appearance progressively.
arXiv Detail & Related papers (2020-07-17T15:58:05Z) - Pose Manipulation with Identity Preservation [0.0]
We introduce Character Adaptive Identity Normalization GAN (CainGAN) which uses spatial characteristic features extracted by an embedder and combined across source images.
CainGAN receives figures of faces from a certain individual and produces new ones while preserving the person's identity.
Experimental results show that the quality of generated images scales with the size of the input set used during inference.
arXiv Detail & Related papers (2020-04-20T09:51:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.