S2FGAN: Semantically Aware Interactive Sketch-to-Face Translation
- URL: http://arxiv.org/abs/2011.14785v1
- Date: Mon, 30 Nov 2020 13:42:39 GMT
- Title: S2FGAN: Semantically Aware Interactive Sketch-to-Face Translation
- Authors: Yan Yang and Md Zakir Hossain and Tom Gedeon and Shafin Rahman
- Abstract summary: This paper proposes a sketch-to-image generation framework called S2FGAN.
We employ two latent spaces to control the face appearance and adjust the desired attributes of the generated face.
Our method successfully outperforms state-of-the-art methods on attribute manipulation by exploiting greater control of attribute intensity.
- Score: 11.724779328025589
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Interactive facial image manipulation attempts to edit single and multiple
face attributes using a photo-realistic face and/or semantic mask as input. In
the absence of the photo-realistic image (only sketch/mask available), previous
methods only retrieve the original face but ignore the potential of aiding
model controllability and diversity in the translation process. This paper
proposes a sketch-to-image generation framework called S2FGAN, aiming to
improve users' ability to interpret and flexibility of face attribute editing
from a simple sketch. The proposed framework modifies the constrained latent
space semantics trained on Generative Adversarial Networks (GANs). We employ
two latent spaces to control the face appearance and adjust the desired
attributes of the generated face. Instead of constraining the translation
process by using a reference image, the users can command the model to retouch
the generated images by involving the semantic information in the generation
process. In this way, our method can manipulate single or multiple face
attributes by only specifying attributes to be changed. Extensive experimental
results on CelebAMask-HQ dataset empirically shows our superior performance and
effectiveness on this task. Our method successfully outperforms
state-of-the-art methods on attribute manipulation by exploiting greater
control of attribute intensity.
Related papers
- FlashFace: Human Image Personalization with High-fidelity Identity Preservation [59.76645602354481]
FlashFace allows users to easily personalize their own photos by providing one or a few reference face images and a text prompt.
Our approach is distinguishable from existing human photo customization methods by higher-fidelity identity preservation and better instruction following.
arXiv Detail & Related papers (2024-03-25T17:59:57Z) - When StyleGAN Meets Stable Diffusion: a $\mathscr{W}_+$ Adapter for
Personalized Image Generation [60.305112612629465]
Text-to-image diffusion models have excelled in producing diverse, high-quality, and photo-realistic images.
We present a novel use of the extended StyleGAN embedding space $mathcalW_+$ to achieve enhanced identity preservation and disentanglement for diffusion models.
Our method adeptly generates personalized text-to-image outputs that are not only compatible with prompt descriptions but also amenable to common StyleGAN editing directions.
arXiv Detail & Related papers (2023-11-29T09:05:14Z) - GaFET: Learning Geometry-aware Facial Expression Translation from
In-The-Wild Images [55.431697263581626]
We introduce a novel Geometry-aware Facial Expression Translation framework, which is based on parametric 3D facial representations and can stably decoupled expression.
We achieve higher-quality and more accurate facial expression transfer results compared to state-of-the-art methods, and demonstrate applicability of various poses and complex textures.
arXiv Detail & Related papers (2023-08-07T09:03:35Z) - DreamIdentity: Improved Editability for Efficient Face-identity
Preserved Image Generation [69.16517915592063]
We propose a novel face-identity encoder to learn an accurate representation of human faces.
We also propose self-augmented editability learning to enhance the editability of models.
Our methods can generate identity-preserved images under different scenes at a much faster speed.
arXiv Detail & Related papers (2023-07-01T11:01:17Z) - ChatFace: Chat-Guided Real Face Editing via Diffusion Latent Space
Manipulation [22.724306705927095]
We propose a novel approach that conduct text-driven image editing in the semantic latent space of diffusion model.
By aligning the temporal feature of the diffusion model with the semantic condition at generative process, we introduce a stable manipulation strategy.
We develop an interactive system named ChatFace, which combines the zero-shot reasoning ability of large language models to perform efficient manipulations.
arXiv Detail & Related papers (2023-05-24T05:28:37Z) - ManiCLIP: Multi-Attribute Face Manipulation from Text [104.30600573306991]
We present a novel multi-attribute face manipulation method based on textual descriptions.
Our method generates natural manipulated faces with minimal text-irrelevant attribute editing.
arXiv Detail & Related papers (2022-10-02T07:22:55Z) - IA-FaceS: A Bidirectional Method for Semantic Face Editing [8.19063619210761]
This paper proposes a bidirectional method for disentangled face attribute manipulation as well as flexible, controllable component editing.
IA-FaceS is developed for the first time without any input visual guidance, such as segmentation masks or sketches.
Both quantitative and qualitative results indicate that the proposed method outperforms the other techniques in reconstruction, face attribute manipulation, and component transfer.
arXiv Detail & Related papers (2022-03-24T14:44:56Z) - FacialGAN: Style Transfer and Attribute Manipulation on Synthetic Faces [9.664892091493586]
FacialGAN is a novel framework enabling simultaneous rich style transfers and interactive facial attributes manipulation.
We show our model's capacity in producing visually compelling results in style transfer, attribute manipulation, diversity and face verification.
arXiv Detail & Related papers (2021-10-18T15:53:38Z) - FaceController: Controllable Attribute Editing for Face in the Wild [74.56117807309576]
We propose a simple feed-forward network to generate high-fidelity manipulated faces.
By simply employing some existing and easy-obtainable prior information, our method can control, transfer, and edit diverse attributes of faces in the wild.
In our method, we decouple identity, expression, pose, and illumination using 3D priors; separate texture and colors by using region-wise style codes.
arXiv Detail & Related papers (2021-02-23T02:47:28Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.