Toward Fine-grained Facial Expression Manipulation

• URL: http://arxiv.org/abs/2004.03132v2
• Date: Fri, 4 Dec 2020 08:19:55 GMT
• Title: Toward Fine-grained Facial Expression Manipulation
• Authors: Jun Ling, Han Xue, Li Song, Shuhui Yang, Rong Xie, Xiao Gu
• Abstract summary: Previous methods edit an input image under the guidance of a discrete emotion label or absolute condition to possess the desired expression. We replace continuous absolute condition with relative condition, specifically, relative action units. With relative action units, the generator learns to only transform regions of interest which are specified by non-zero-valued relative AUs.
• Score: 20.226370494178617
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Facial expression manipulation aims at editing facial expression with a given condition. Previous methods edit an input image under the guidance of a discrete emotion label or absolute condition (e.g., facial action units) to possess the desired expression. However, these methods either suffer from changing condition-irrelevant regions or are inefficient for fine-grained editing. In this study, we take these two objectives into consideration and propose a novel method. First, we replace continuous absolute condition with relative condition, specifically, relative action units. With relative action units, the generator learns to only transform regions of interest which are specified by non-zero-valued relative AUs. Second, our generator is built on U-Net but strengthened by Multi-Scale Feature Fusion (MSF) mechanism for high-quality expression editing purposes. Extensive experiments on both quantitative and qualitative evaluation demonstrate the improvements of our proposed approach compared to the state-of-the-art expression editing methods. Code is available at \url{https://github.com/junleen/Expression-manipulator}.

Related papers

• Knowledge-Enhanced Facial Expression Recognition with Emotional-to-Neutral Transformation [66.53435569574135]
  Existing facial expression recognition methods typically fine-tune a pre-trained visual encoder using discrete labels. We observe that the rich knowledge in text embeddings, generated by vision-language models, is a promising alternative for learning discriminative facial expression representations. We propose a novel knowledge-enhanced FER method with an emotional-to-neutral transformation.
  arXiv  Detail & Related papers  (2024-09-13T07:28:57Z)
• Towards Localized Fine-Grained Control for Facial Expression Generation [54.82883891478555]
  Humans, particularly their faces, are central to content generation due to their ability to convey rich expressions and intent. Current generative models mostly generate flat neutral expressions and characterless smiles without authenticity. We propose the use of AUs (action units) for facial expression control in face generation.
  arXiv  Detail & Related papers  (2024-07-25T18:29:48Z)
• GaFET: Learning Geometry-aware Facial Expression Translation from In-The-Wild Images [55.431697263581626]
  We introduce a novel Geometry-aware Facial Expression Translation framework, which is based on parametric 3D facial representations and can stably decoupled expression. We achieve higher-quality and more accurate facial expression transfer results compared to state-of-the-art methods, and demonstrate applicability of various poses and complex textures.
  arXiv  Detail & Related papers  (2023-08-07T09:03:35Z)
• FEAT: Face Editing with Attention [70.89233432407305]
  We build on the StyleGAN generator and present a method that explicitly encourages face manipulation to focus on the intended regions. During the generation of the edited image, the attention map serves as a mask that guides a blending between the original features and the modified ones.
  arXiv  Detail & Related papers  (2022-02-06T06:07:34Z)
• Perceptually Validated Precise Local Editing for Facial Action Units with StyleGAN [3.8149289266694466]
  We build a solution based on StyleGAN, which has been used extensively for semantic manipulation of faces. We show that a naive strategy to perform editing in the latent space results in undesired coupling between certain action units. We validate the effectiveness of our local editing method through perception experiments conducted with 23 subjects.
  arXiv  Detail & Related papers  (2021-07-26T12:21:37Z)
• Detection and Localization of Facial Expression Manipulations [44.52966548652561]
  We propose a framework that is able to detect manipulations in facial expression using a close combination of facial expression recognition and image manipulation methods. We show that, on the Face2Face dataset, where there is abundant expression manipulation, our method achieves over 3% higher accuracy for both classification and localization of manipulations.
  arXiv  Detail & Related papers  (2021-03-15T04:35:56Z)
• S2FGAN: Semantically Aware Interactive Sketch-to-Face Translation [11.724779328025589]
  This paper proposes a sketch-to-image generation framework called S2FGAN. We employ two latent spaces to control the face appearance and adjust the desired attributes of the generated face. Our method successfully outperforms state-of-the-art methods on attribute manipulation by exploiting greater control of attribute intensity.
  arXiv  Detail & Related papers  (2020-11-30T13:42:39Z)
• LEED: Label-Free Expression Editing via Disentanglement [57.09545215087179]
  LEED framework is capable of editing the expression of both frontal and profile facial images without requiring any expression label. Two novel losses are designed for optimal expression disentanglement and consistent synthesis.
  arXiv  Detail & Related papers  (2020-07-17T13:36:15Z)
• Cascade EF-GAN: Progressive Facial Expression Editing with Local Focuses [49.077232276128754]
  We propose a novel network that performs progressive facial expression editing with local expression focuses. The introduction of the local focus enables the Cascade EF-GAN to better preserve identity-related features. In addition, an innovative cascade transformation strategy is designed by dividing a large facial expression transformation into multiple small ones in cascade.
  arXiv  Detail & Related papers  (2020-03-12T17:11:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.