LEED: Label-Free Expression Editing via Disentanglement

• URL: http://arxiv.org/abs/2007.08971v1
• Date: Fri, 17 Jul 2020 13:36:15 GMT
• Title: LEED: Label-Free Expression Editing via Disentanglement
• Authors: Rongliang Wu, Shijian Lu
• Abstract summary: LEED framework is capable of editing the expression of both frontal and profile facial images without requiring any expression label. Two novel losses are designed for optimal expression disentanglement and consistent synthesis.
• Score: 57.09545215087179
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent studies on facial expression editing have obtained very promising progress. On the other hand, existing methods face the constraint of requiring a large amount of expression labels which are often expensive and time-consuming to collect. This paper presents an innovative label-free expression editing via disentanglement (LEED) framework that is capable of editing the expression of both frontal and profile facial images without requiring any expression label. The idea is to disentangle the identity and expression of a facial image in the expression manifold, where the neutral face captures the identity attribute and the displacement between the neutral image and the expressive image captures the expression attribute. Two novel losses are designed for optimal expression disentanglement and consistent synthesis, including a mutual expression information loss that aims to extract pure expression-related features and a siamese loss that aims to enhance the expression similarity between the synthesized image and the reference image. Extensive experiments over two public facial expression datasets show that LEED achieves superior facial expression editing qualitatively and quantitatively.

Related papers

• Knowledge-Enhanced Facial Expression Recognition with Emotional-to-Neutral Transformation [66.53435569574135]
  Existing facial expression recognition methods typically fine-tune a pre-trained visual encoder using discrete labels. We observe that the rich knowledge in text embeddings, generated by vision-language models, is a promising alternative for learning discriminative facial expression representations. We propose a novel knowledge-enhanced FER method with an emotional-to-neutral transformation.
  arXiv  Detail & Related papers  (2024-09-13T07:28:57Z)
• Towards a Simultaneous and Granular Identity-Expression Control in Personalized Face Generation [34.72612800373437]
  In human-centric content generation, pre-trained text-to-image models struggle to produce user-wanted portrait images. We propose a novel multi-modal face generation framework, capable of simultaneous identity-expression control and more fine-grained expression synthesis.
  arXiv  Detail & Related papers  (2024-01-02T13:28:39Z)
• When StyleGAN Meets Stable Diffusion: a $\mathscr{W}_+$ Adapter for Personalized Image Generation [60.305112612629465]
  Text-to-image diffusion models have excelled in producing diverse, high-quality, and photo-realistic images. We present a novel use of the extended StyleGAN embedding space $mathcalW_+$ to achieve enhanced identity preservation and disentanglement for diffusion models. Our method adeptly generates personalized text-to-image outputs that are not only compatible with prompt descriptions but also amenable to common StyleGAN editing directions.
  arXiv  Detail & Related papers  (2023-11-29T09:05:14Z)
• GaFET: Learning Geometry-aware Facial Expression Translation from In-The-Wild Images [55.431697263581626]
  We introduce a novel Geometry-aware Facial Expression Translation framework, which is based on parametric 3D facial representations and can stably decoupled expression. We achieve higher-quality and more accurate facial expression transfer results compared to state-of-the-art methods, and demonstrate applicability of various poses and complex textures.
  arXiv  Detail & Related papers  (2023-08-07T09:03:35Z)
• Disentangling Identity and Pose for Facial Expression Recognition [54.50747989860957]
  We propose an identity and pose disentangled facial expression recognition (IPD-FER) model to learn more discriminative feature representation. For identity encoder, a well pre-trained face recognition model is utilized and fixed during training, which alleviates the restriction on specific expression training data. By comparing the difference between synthesized neutral and expressional images of the same individual, the expression component is further disentangled from identity and pose.
  arXiv  Detail & Related papers  (2022-08-17T06:48:13Z)
• Facial Expression Editing with Continuous Emotion Labels [76.36392210528105]
  Deep generative models have achieved impressive results in the field of automated facial expression editing. We propose a model that can be used to manipulate facial expressions in facial images according to continuous two-dimensional emotion labels.
  arXiv  Detail & Related papers  (2020-06-22T13:03:02Z)
• Fine-Grained Expression Manipulation via Structured Latent Space [30.789513209376032]
  We propose an end-to-end expression-guided generative adversarial network (EGGAN) to manipulate fine-grained expressions. Our method can manipulate fine-grained expressions, and generate continuous intermediate expressions between source and target expressions.
  arXiv  Detail & Related papers  (2020-04-21T06:18:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.