FEAT: Face Editing with Attention

• URL: http://arxiv.org/abs/2202.02713v1
• Date: Sun, 6 Feb 2022 06:07:34 GMT
• Title: FEAT: Face Editing with Attention
• Authors: Xianxu Hou, Linlin Shen, Or Patashnik, Daniel Cohen-Or, Hui Huang
• Abstract summary: We build on the StyleGAN generator and present a method that explicitly encourages face manipulation to focus on the intended regions. During the generation of the edited image, the attention map serves as a mask that guides a blending between the original features and the modified ones.
• Score: 70.89233432407305
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Employing the latent space of pretrained generators has recently been shown to be an effective means for GAN-based face manipulation. The success of this approach heavily relies on the innate disentanglement of the latent space axes of the generator. However, face manipulation often intends to affect local regions only, while common generators do not tend to have the necessary spatial disentanglement. In this paper, we build on the StyleGAN generator, and present a method that explicitly encourages face manipulation to focus on the intended regions by incorporating learned attention maps. During the generation of the edited image, the attention map serves as a mask that guides a blending between the original features and the modified ones. The guidance for the latent space edits is achieved by employing CLIP, which has recently been shown to be effective for text-driven edits. We perform extensive experiments and show that our method can perform disentangled and controllable face manipulations based on text descriptions by attending to the relevant regions only. Both qualitative and quantitative experimental results demonstrate the superiority of our method for facial region editing over alternative methods.

Related papers

• Revealing Directions for Text-guided 3D Face Editing [52.85632020601518]
  3D face editing is a significant task in multimedia, aimed at the manipulation of 3D face models across various control signals. We present Face Clan, a text-general approach for generating and manipulating 3D faces based on arbitrary attribute descriptions. Our method offers a precisely controllable manipulation method, allowing users to intuitively customize regions of interest with the text description.
  arXiv  Detail & Related papers  (2024-10-07T12:04:39Z)
• ZONE: Zero-Shot Instruction-Guided Local Editing [56.56213730578504]
  We propose a Zero-shot instructiON-guided local image Editing approach, termed ZONE. We first convert the editing intent from the user-provided instruction into specific image editing regions through InstructPix2Pix. We then propose a Region-IoU scheme for precise image layer extraction from an off-the-shelf segment model.
  arXiv  Detail & Related papers  (2023-12-28T02:54:34Z)
• LC-NeRF: Local Controllable Face Generation in Neural Randiance Field [55.54131820411912]
  LC-NeRF is composed of a Local Region Generators Module and a Spatial-Aware Fusion Module. Our method provides better local editing than state-of-the-art face editing methods. Our method also performs well in downstream tasks, such as text-driven facial image editing.
  arXiv  Detail & Related papers  (2023-02-19T05:50:08Z)
• Video2StyleGAN: Disentangling Local and Global Variations in a Video [68.70889857355678]
  StyleGAN has emerged as a powerful paradigm for facial editing, providing disentangled controls over age, expression, illumination, etc. We introduce Video2StyleGAN that takes a target image and driving video(s) to reenact the local and global locations and expressions from the driving video in the identity of the target image.
  arXiv  Detail & Related papers  (2022-05-27T14:18:19Z)
• Perceptually Validated Precise Local Editing for Facial Action Units with StyleGAN [3.8149289266694466]
  We build a solution based on StyleGAN, which has been used extensively for semantic manipulation of faces. We show that a naive strategy to perform editing in the latent space results in undesired coupling between certain action units. We validate the effectiveness of our local editing method through perception experiments conducted with 23 subjects.
  arXiv  Detail & Related papers  (2021-07-26T12:21:37Z)
• Pivotal Tuning for Latent-based Editing of Real Images [40.22151052441958]
  A surge of advanced facial editing techniques have been proposed that leverage the generative power of a pre-trained StyleGAN. To successfully edit an image this way, one must first project (or invert) the image into the pre-trained generator's domain. This means it is still challenging to apply ID-preserving facial latent-space editing to faces which are out of the generator's domain.
  arXiv  Detail & Related papers  (2021-06-10T13:47:59Z)
• FaceController: Controllable Attribute Editing for Face in the Wild [74.56117807309576]
  We propose a simple feed-forward network to generate high-fidelity manipulated faces. By simply employing some existing and easy-obtainable prior information, our method can control, transfer, and edit diverse attributes of faces in the wild. In our method, we decouple identity, expression, pose, and illumination using 3D priors; separate texture and colors by using region-wise style codes.
  arXiv  Detail & Related papers  (2021-02-23T02:47:28Z)
• Toward Fine-grained Facial Expression Manipulation [20.226370494178617]
  Previous methods edit an input image under the guidance of a discrete emotion label or absolute condition to possess the desired expression. We replace continuous absolute condition with relative condition, specifically, relative action units. With relative action units, the generator learns to only transform regions of interest which are specified by non-zero-valued relative AUs.
  arXiv  Detail & Related papers  (2020-04-07T05:14:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.