MyStyle++: A Controllable Personalized Generative Prior
- URL: http://arxiv.org/abs/2306.04865v3
- Date: Tue, 10 Oct 2023 22:56:41 GMT
- Title: MyStyle++: A Controllable Personalized Generative Prior
- Authors: Libing Zeng, Lele Chen, Yi Xu, Nima Kalantari
- Abstract summary: We build upon MyStyle, a recently introduced method, that tunes the weights of a pre-trained StyleGAN face generator on a few images of an individual.
MyStyle does not demonstrate precise control over the attributes of the generated images.
We demonstrate that our approach, dubbed MyStyle++, is able to synthesize, edit, and enhance images of an individual with great control over the attributes.
- Score: 13.030741541265428
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In this paper, we propose an approach to obtain a personalized generative
prior with explicit control over a set of attributes. We build upon MyStyle, a
recently introduced method, that tunes the weights of a pre-trained StyleGAN
face generator on a few images of an individual. This system allows
synthesizing, editing, and enhancing images of the target individual with high
fidelity to their facial features. However, MyStyle does not demonstrate
precise control over the attributes of the generated images. We propose to
address this problem through a novel optimization system that organizes the
latent space in addition to tuning the generator. Our key contribution is to
formulate a loss that arranges the latent codes, corresponding to the input
images, along a set of specific directions according to their attributes. We
demonstrate that our approach, dubbed MyStyle++, is able to synthesize, edit,
and enhance images of an individual with great control over the attributes,
while preserving the unique facial characteristics of that individual.
Related papers
- DisEnvisioner: Disentangled and Enriched Visual Prompt for Customized Image Generation [22.599542105037443]
DisEnvisioner is a novel approach for effectively extracting and enriching the subject-essential features while filtering out -irrelevant information.
Specifically, the feature of the subject and other irrelevant components are effectively separated into distinctive visual tokens, enabling a much more accurate customization.
Experiments demonstrate the superiority of our approach over existing methods in instruction response (editability), ID consistency, inference speed, and the overall image quality.
arXiv Detail & Related papers (2024-10-02T22:29:14Z) - OSTAF: A One-Shot Tuning Method for Improved Attribute-Focused T2I Personalization [9.552325786494334]
We introduce a novel parameter-efficient one-shot fine-tuning method for personalized text-to-image (T2I) personalization.
A novel hypernetwork-powered attribute-focused fine-tuning mechanism is employed to achieve the precise learning of various attribute features.
Our method shows significant superiority in attribute identification and application, as well as achieves a good balance between efficiency and output quality.
arXiv Detail & Related papers (2024-03-17T01:42:48Z) - PALP: Prompt Aligned Personalization of Text-to-Image Models [68.91005384187348]
Existing personalization methods compromise personalization ability or the alignment to complex prompts.
We propose a new approach focusing on personalization methods for a emphsingle prompt to address this issue.
Our method excels in improving text alignment, enabling the creation of images with complex and intricate prompts.
arXiv Detail & Related papers (2024-01-11T18:35:33Z) - When StyleGAN Meets Stable Diffusion: a $\mathscr{W}_+$ Adapter for
Personalized Image Generation [60.305112612629465]
Text-to-image diffusion models have excelled in producing diverse, high-quality, and photo-realistic images.
We present a novel use of the extended StyleGAN embedding space $mathcalW_+$ to achieve enhanced identity preservation and disentanglement for diffusion models.
Our method adeptly generates personalized text-to-image outputs that are not only compatible with prompt descriptions but also amenable to common StyleGAN editing directions.
arXiv Detail & Related papers (2023-11-29T09:05:14Z) - StyleSync: High-Fidelity Generalized and Personalized Lip Sync in
Style-based Generator [85.40502725367506]
We propose StyleSync, an effective framework that enables high-fidelity lip synchronization.
Specifically, we design a mask-guided spatial information encoding module that preserves the details of the given face.
Our design also enables personalized lip-sync by introducing style space and generator refinement on only limited frames.
arXiv Detail & Related papers (2023-05-09T13:38:13Z) - Everything is There in Latent Space: Attribute Editing and Attribute
Style Manipulation by StyleGAN Latent Space Exploration [39.18239951479647]
We present Few-shot Latent-based Attribute Manipulation and Editing (FLAME)
FLAME is a framework to perform highly controlled image editing by latent space manipulation.
We generate diverse attribute styles in disentangled manner.
arXiv Detail & Related papers (2022-07-20T12:40:32Z) - MyStyle: A Personalized Generative Prior [38.3436972491162]
We introduce MyStyle, a personalized deep generative prior trained with a few shots of an individual.
MyStyle allows to reconstruct, enhance and edit images of a specific person.
arXiv Detail & Related papers (2022-03-31T17:59:19Z) - AttrLostGAN: Attribute Controlled Image Synthesis from Reconfigurable
Layout and Style [5.912209564607099]
We propose a method for attribute controlled image synthesis from layout.
We extend a state-of-the-art approach for layout-to-image generation to condition individual objects on attributes.
Our results show that our method can successfully control the fine-grained details of individual objects when modelling complex scenes with multiple objects.
arXiv Detail & Related papers (2021-03-25T10:09:45Z) - FaceController: Controllable Attribute Editing for Face in the Wild [74.56117807309576]
We propose a simple feed-forward network to generate high-fidelity manipulated faces.
By simply employing some existing and easy-obtainable prior information, our method can control, transfer, and edit diverse attributes of faces in the wild.
In our method, we decouple identity, expression, pose, and illumination using 3D priors; separate texture and colors by using region-wise style codes.
arXiv Detail & Related papers (2021-02-23T02:47:28Z) - PIE: Portrait Image Embedding for Semantic Control [82.69061225574774]
We present the first approach for embedding real portrait images in the latent space of StyleGAN.
We use StyleRig, a pretrained neural network that maps the control space of a 3D morphable face model to the latent space of the GAN.
An identity energy preservation term allows spatially coherent edits while maintaining facial integrity.
arXiv Detail & Related papers (2020-09-20T17:53:51Z) - Generating Person Images with Appearance-aware Pose Stylizer [66.44220388377596]
We present a novel end-to-end framework to generate realistic person images based on given person poses and appearances.
The core of our framework is a novel generator called Appearance-aware Pose Stylizer (APS) which generates human images by coupling the target pose with the conditioned person appearance progressively.
arXiv Detail & Related papers (2020-07-17T15:58:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.