Giving Faces Their Feelings Back: Explicit Emotion Control for Feedforward Single-Image 3D Head Avatars

• URL: http://arxiv.org/abs/2604.14541v1
• Date: Thu, 16 Apr 2026 02:16:36 GMT
• Title: Giving Faces Their Feelings Back: Explicit Emotion Control for Feedforward Single-Image 3D Head Avatars
• Abstract summary: We present a framework for explicit emotion control in feed-forward, single-image 3D head avatar reconstruction.<n>We treat emotion as a first-class control signal that can be manipulated independently and consistently across identities.
• Score: 30.195753051123003
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a framework for explicit emotion control in feed-forward, single-image 3D head avatar reconstruction. Unlike existing pipelines where emotion is implicitly entangled with geometry or appearance, we treat emotion as a first-class control signal that can be manipulated independently and consistently across identities. Our method injects emotion into existing feed-forward architectures via a dual-path modulation mechanism without modifying their core design. Geometry modulation performs emotion-conditioned normalization in the original parametric space, disentangling emotional state from speech-driven articulation, while appearance modulation captures identity-aware, emotion-dependent visual cues beyond geometry. To enable learning under this setting, we construct a time-synchronized, emotion-consistent multi-identity dataset by transferring aligned emotional dynamics across identities. Integrated into multiple state-of-the-art backbones, our framework preserves reconstruction and reenactment fidelity while enabling controllable emotion transfer, disentangled manipulation, and smooth emotion interpolation, advancing expressive and scalable 3D head avatars.

Related papers

• Geometry-guided Emotion Modulation for Controllable and Photorealistic Emotional Talking Face Generation [47.04292510603205]
  GemTalk is a diffusion-based framework that combines the semantic richness of implicit representations with the structural precision of explicit geometric priors.<n>We introduce a Vision-guided Audio Emotion Projection (V-AEP) module to extract implicit emotional lip and expression features.<n>At the same time, a Diffusion-based Geometric Priors Generator (D-GPG) generates identity-aware blend coefficients as explicit structural priors.
  arXiv  Detail & Related papers  (2026-08-01T13:33:38Z)
• SEDTalker: Emotion-Aware 3D Facial Animation Using Frame-Level Speech Emotion Diarization [14.632533340477591]
  SEDTalker is an emotion-aware framework for speech-driven 3D facial animation.<n>We use frame-level speech emotion diarization to achieve fine-grained expressive control.
  arXiv  Detail & Related papers  (2026-04-14T22:55:09Z)
• EmoKGEdit: Training-free Affective Injection via Visual Cue Transformation [7.245162028678732]
  EmoKGEdit is a novel training-free framework for precise and structure-preserving image emotion editing.<n>We construct a Multimodal Sentiment Association Knowledge Graph to disentangle the relationships among objects, scenes, attributes, visual clues and emotion.<n>EmoKGEdit achieves excellent performance in both emotion fidelity and content preservation, and outperforms the state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-01-18T09:20:09Z)
• Mind-to-Face: Neural-Driven Photorealistic Avatar Synthesis via EEG Decoding [11.030344145348097]
  We present Mind-to-Face, the first framework that decodes non-invasive electroencephalogram (EEG) signals directly into high-fidelity facial expressions.<n>We show that EEG alone can reliably predict dynamic, subject-specific facial expressions, including subtle emotional responses.<n>Mind-to-Face establishes a new paradigm for neural-driven avatars, enabling personalized, emotion-aware telepresence and cognitive interaction in immersive environments.
  arXiv  Detail & Related papers  (2025-12-03T23:02:27Z)
• MEDTalk: Multimodal Controlled 3D Facial Animation with Dynamic Emotions by Disentangled Embedding [48.54455964043634]
  MEDTalk is a novel framework for fine-grained and dynamic emotional talking head generation.<n>We integrate audio and speech text, predicting frame-wise intensity variations and dynamically adjusting static emotion features to generate realistic emotional expressions.<n>Our generated results can be conveniently integrated into the industrial production pipeline.
  arXiv  Detail & Related papers  (2025-07-08T15:14:27Z)
• UDDETTS: Unifying Discrete and Dimensional Emotions for Controllable Emotional Text-to-Speech [61.989360995528905]
  We propose UDDETTS, a universal framework unifying discrete and dimensional emotions for controllable emotional TTS.<n>This model introduces the interpretable Arousal-Dominance-Valence (ADV) space for dimensional emotion description and supports emotion control driven by either discrete emotion labels or nonlinearly quantified ADV values.<n>Experiments show that UDDETTS achieves linear emotion control along three interpretable dimensions, and exhibits superior end-to-end emotional speech synthesis capabilities.
  arXiv  Detail & Related papers  (2025-05-15T12:57:19Z)
• Disentangle Identity, Cooperate Emotion: Correlation-Aware Emotional Talking Portrait Generation [63.94836524433559]
  DICE-Talk is a framework for disentangling identity with emotion and cooperating emotions with similar characteristics.<n>We develop a disentangled emotion embedder that jointly models audio-visual emotional cues through cross-modal attention.<n>Second, we introduce a correlation-enhanced emotion conditioning module with learnable Emotion Banks.<n>Third, we design an emotion discrimination objective that enforces affective consistency during the diffusion process.
  arXiv  Detail & Related papers  (2025-04-25T05:28:21Z)
• Controllable Expressive 3D Facial Animation via Diffusion in a Unified Multimodal Space [7.165879904419689]
  We present a diffusion-based framework for controllable expressive 3D facial animation.<n>Our approach introduces two key innovations: (1) a FLAME-centered multimodal emotion binding strategy, and (2) an attention-based latent diffusion model.<n>Our method achieves a 21.6% improvement in emotion similarity while preserving physiologically plausible facial dynamics.
  arXiv  Detail & Related papers  (2025-04-14T01:38:01Z)
• GMTalker: Gaussian Mixture-based Audio-Driven Emotional Talking Video Portraits [60.05683966405544]
  We present GMTalker, a Gaussian mixture-based emotional talking portraits generation framework.<n>Specifically, we propose a continuous and disentangled latent space, achieving more flexible emotion manipulation.<n>We also introduce a normalizing flow-based motion generator pretrained on a large dataset to generate diverse head poses, blinks, and eyeball movements.
  arXiv  Detail & Related papers  (2023-12-12T19:03:04Z)
• Weakly-Supervised Emotion Transition Learning for Diverse 3D Co-speech Gesture Generation [43.04371187071256]
  We present a novel method to generate vivid and emotional 3D co-speech gestures in 3D avatars. We use the ChatGPT-4 and an audio inpainting approach to construct the high-fidelity emotion transition human speeches. Our method outperforms the state-of-the-art models constructed by adapting single emotion-conditioned counterparts.
  arXiv  Detail & Related papers  (2023-11-29T11:10:40Z)
• High-fidelity Generalized Emotional Talking Face Generation with Multi-modal Emotion Space Learning [43.09015109281053]
  We propose a more flexible and generalized framework for talking face generation. Specifically, we supplement the emotion style in text prompts and use an Aligned Multi-modal Emotion encoder to embed the text, image, and audio emotion modality into a unified space. An Emotion-aware Audio-to-3DMM Convertor is proposed to connect the emotion condition and the audio sequence to structural representation.
  arXiv  Detail & Related papers  (2023-05-04T05:59:34Z)
• Enhancing Cognitive Models of Emotions with Representation Learning [58.2386408470585]
  We present a novel deep learning-based framework to generate embedding representations of fine-grained emotions. Our framework integrates a contextualized embedding encoder with a multi-head probing model. Our model is evaluated on the Empathetic Dialogue dataset and shows the state-of-the-art result for classifying 32 emotions.
  arXiv  Detail & Related papers  (2021-04-20T16:55:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.