Altering Facial Expression Based on Textual Emotion

• URL: http://arxiv.org/abs/2112.01454v1
• Date: Thu, 2 Dec 2021 17:52:25 GMT
• Title: Altering Facial Expression Based on Textual Emotion
• Authors: Mohammad Imrul Jubair, Md. Masud Rana, Md. Amir Hamza, Mohsena Ashraf, Fahim Ahsan Khan, Ahnaf Tahseen Prince
• Abstract summary: We aim to change the facial expression in an image using the Generative Adversarial Network (GAN) We extend our work by remodeling facial expressions in an image indicated by the emotion from a given text. As a demonstration of our working pipeline, we create an application prototype of a blog that regenerates the profile picture with different expressions.
• Score: 0.19573380763700707
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Faces and their expressions are one of the potent subjects for digital images. Detecting emotions from images is an ancient task in the field of computer vision; however, performing its reverse -- synthesizing facial expressions from images -- is quite new. Such operations of regenerating images with different facial expressions, or altering an existing expression in an image require the Generative Adversarial Network (GAN). In this paper, we aim to change the facial expression in an image using GAN, where the input image with an initial expression (i.e., happy) is altered to a different expression (i.e., disgusted) for the same person. We used StarGAN techniques on a modified version of the MUG dataset to accomplish this objective. Moreover, we extended our work further by remodeling facial expressions in an image indicated by the emotion from a given text. As a result, we applied a Long Short-Term Memory (LSTM) method to extract emotion from the text and forwarded it to our expression-altering module. As a demonstration of our working pipeline, we also create an application prototype of a blog that regenerates the profile picture with different expressions based on the user's textual emotion.

Related papers

• EmotiCrafter: Text-to-Emotional-Image Generation based on Valence-Arousal Model [23.26111054485357]
  We introduce the new task of continuous emotional image content generation (C-EICG) We present EmotiCrafter, an emotional image generation model that generates images based on text prompts and Valence-Arousal values.
  arXiv  Detail & Related papers  (2025-01-10T04:41:37Z)
• Knowledge-Enhanced Facial Expression Recognition with Emotional-to-Neutral Transformation [66.53435569574135]
  Existing facial expression recognition methods typically fine-tune a pre-trained visual encoder using discrete labels. We observe that the rich knowledge in text embeddings, generated by vision-language models, is a promising alternative for learning discriminative facial expression representations. We propose a novel knowledge-enhanced FER method with an emotional-to-neutral transformation.
  arXiv  Detail & Related papers  (2024-09-13T07:28:57Z)
• Towards Localized Fine-Grained Control for Facial Expression Generation [54.82883891478555]
  Humans, particularly their faces, are central to content generation due to their ability to convey rich expressions and intent. Current generative models mostly generate flat neutral expressions and characterless smiles without authenticity. We propose the use of AUs (action units) for facial expression control in face generation.
  arXiv  Detail & Related papers  (2024-07-25T18:29:48Z)
• GaFET: Learning Geometry-aware Facial Expression Translation from In-The-Wild Images [55.431697263581626]
  We introduce a novel Geometry-aware Facial Expression Translation framework, which is based on parametric 3D facial representations and can stably decoupled expression. We achieve higher-quality and more accurate facial expression transfer results compared to state-of-the-art methods, and demonstrate applicability of various poses and complex textures.
  arXiv  Detail & Related papers  (2023-08-07T09:03:35Z)
• RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions [9.51095076299351]
  We develop RePrompt, an automatic method to refine text prompts toward precise expression of generated images. Inspired by crowdsourced editing strategies, we curated intuitive text features, such as the number and concreteness of nouns. With model explanations of the proxy model, we curated a rubric to adjust text prompts to optimize image generation for precise emotion expression.
  arXiv  Detail & Related papers  (2023-02-19T03:31:31Z)
• Emotion Separation and Recognition from a Facial Expression by Generating the Poker Face with Vision Transformers [57.1091606948826]
  We propose a novel FER model, named Poker Face Vision Transformer or PF-ViT, to address these challenges. PF-ViT aims to separate and recognize the disturbance-agnostic emotion from a static facial image via generating its corresponding poker face. PF-ViT utilizes vanilla Vision Transformers, and its components are pre-trained as Masked Autoencoders on a large facial expression dataset.
  arXiv  Detail & Related papers  (2022-07-22T13:39:06Z)
• Emotion-Controllable Generalized Talking Face Generation [6.22276955954213]
  We propose a one-shot facial geometry-aware emotional talking face generation method. Our method can adapt to arbitrary faces captured in-the-wild by fine-tuning with only a single image of the target identity in neutral emotion.
  arXiv  Detail & Related papers  (2022-05-02T18:41:36Z)
• LEED: Label-Free Expression Editing via Disentanglement [57.09545215087179]
  LEED framework is capable of editing the expression of both frontal and profile facial images without requiring any expression label. Two novel losses are designed for optimal expression disentanglement and consistent synthesis.
  arXiv  Detail & Related papers  (2020-07-17T13:36:15Z)
• Facial Expression Editing with Continuous Emotion Labels [76.36392210528105]
  Deep generative models have achieved impressive results in the field of automated facial expression editing. We propose a model that can be used to manipulate facial expressions in facial images according to continuous two-dimensional emotion labels.
  arXiv  Detail & Related papers  (2020-06-22T13:03:02Z)
• Real-time Facial Expression Recognition "In The Wild'' by Disentangling 3D Expression from Identity [6.974241731162878]
  This paper proposes a novel method for human emotion recognition from a single RGB image. We construct a large-scale dataset of facial videos, rich in facial dynamics, identities, expressions, appearance and 3D pose variations. Our proposed framework runs at 50 frames per second and is capable of robustly estimating parameters of 3D expression variation.
  arXiv  Detail & Related papers  (2020-05-12T01:32:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.