EmoTaG: Emotion-Aware Talking Head Synthesis on Gaussian Splatting with Few-Shot Personalization

• URL: http://arxiv.org/abs/2603.21332v2
• Date: Sat, 28 Mar 2026 04:29:27 GMT
• Title: EmoTaG: Emotion-Aware Talking Head Synthesis on Gaussian Splatting with Few-Shot Personalization
• Abstract summary: We present EmoTaG, a few-shot emotion-aware 3D talking head synthesis framework built on the Pretrain-and-Adapt paradigm.<n>Our key insight is to reformulate motion prediction in a structured FLAME parameter space rather than directly deforming 3D Gaussians.<n>Building upon this, we propose a Gated Residual Motion Network (GRMN), which captures emotional prosody from audio while supplementing head pose and upper-face cues absent from audio.
• Score: 12.013744751827426
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Audio-driven 3D talking head synthesis has advanced rapidly with Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS). By leveraging rich pre-trained priors, few-shot methods enable instant personalization from just a few seconds of video. However, under expressive facial motion, existing few-shot approaches often suffer from geometric instability and audio-emotion mismatch, highlighting the need for more effective emotion-aware motion modeling. In this work, we present EmoTaG, a few-shot emotion-aware 3D talking head synthesis framework built on the Pretrain-and-Adapt paradigm. Our key insight is to reformulate motion prediction in a structured FLAME parameter space rather than directly deforming 3D Gaussians, thereby introducing explicit geometric priors that improve motion stability. Building upon this, we propose a Gated Residual Motion Network (GRMN), which captures emotional prosody from audio while supplementing head pose and upper-face cues absent from audio, enabling expressive and coherent motion generation. Extensive experiments demonstrate that EmoTaG achieves state-of-the-art performance in emotional expressiveness, lip synchronization, visual realism, and motion stability.

Related papers

• CETalk: Continuous Valence-Arousal Control for Audio-Driven 3D Talking Head Generation [18.841894940869157]
  We propose CETalk, an audio-driven 3D facial animation framework conditioned on Valence--Arousal representations for fine-grained emotion control.<n>We show that CETalk outperforms state-of-the-art methods in both lip-sync accuracy and emotional expressiveness.
  arXiv  Detail & Related papers  (2026-08-15T08:22:50Z)
• GaussianEmoTalker: Real-Time Emotional Talking Head Synthesis with Audio-Driven and Blendshape-Based 3D Gaussian Splatting [28.604834460278706]
  We present an audio-driven framework for real-time emotional talking head synthesis based on 3D Gaussian Splatting.<n>GaussianEmoTalker achieves competitive video quality, accurate lip synchronization, controllable emotional expression, and real-time rendering.
  arXiv  Detail & Related papers  (2026-07-01T13:56:38Z)
• ESGaussianFace: Emotional and Stylized Audio-Driven Facial Animation via 3D Gaussian Splatting [34.65130896150361]
  ESGaussianFace is an innovative framework for emotional and stylized audio-driven facial animation.<n>We propose an emotion-audio-guided spatial attention method that effectively integrates emotion features with audio content features.<n>Our generated results exhibit high efficiency, high quality, and 3D consistency.
  arXiv  Detail & Related papers  (2026-01-05T07:19:38Z)
• EmoCAST: Emotional Talking Portrait via Emotive Text Description [56.42674612728354]
  EmoCAST is a diffusion-based framework for precise text-driven emotional synthesis.<n>In appearance modeling, emotional prompts are integrated through a text-guided decoupled emotive module.<n>EmoCAST achieves state-of-the-art performance in generating realistic, emotionally expressive, and audio-synchronized talking-head videos.
  arXiv  Detail & Related papers  (2025-08-28T10:02:06Z)
• MEDTalk: Multimodal Controlled 3D Facial Animation with Dynamic Emotions by Disentangled Embedding [48.54455964043634]
  MEDTalk is a novel framework for fine-grained and dynamic emotional talking head generation.<n>We integrate audio and speech text, predicting frame-wise intensity variations and dynamically adjusting static emotion features to generate realistic emotional expressions.<n>Our generated results can be conveniently integrated into the industrial production pipeline.
  arXiv  Detail & Related papers  (2025-07-08T15:14:27Z)
• GaussianSpeech: Audio-Driven Gaussian Avatars [76.10163891172192]
  We introduce GaussianSpeech, a novel approach that synthesizes high-fidelity animation sequences of photo-realistic, personalized 3D human head avatars from spoken audio.<n>We propose a compact and efficient 3DGS-based avatar representation that generates expression-dependent color and leverages wrinkle- and perceptually-based losses to synthesize facial details.
  arXiv  Detail & Related papers  (2024-11-27T18:54:08Z)
• DEEPTalk: Dynamic Emotion Embedding for Probabilistic Speech-Driven 3D Face Animation [14.07086606183356]
  Speech-driven 3D facial animation has garnered lots of attention thanks to its broad range of applications.<n>Current methods fail to capture the nuanced emotional undertones conveyed through speech and produce monotonous facial motion.<n>We introduce DEEPTalk, a novel approach that generates diverse and emotionally rich 3D facial expressions directly from speech inputs.
  arXiv  Detail & Related papers  (2024-08-12T08:56:49Z)
• EmoTalk3D: High-Fidelity Free-View Synthesis of Emotional 3D Talking Head [30.138347111341748]
  We present a novel approach for synthesizing 3D talking heads with controllable emotion. Our model enables controllable emotion in the generated talking heads and can be rendered in wide-range views. Experiments demonstrate the effectiveness of our approach in generating high-fidelity and emotion-controllable 3D talking heads.
  arXiv  Detail & Related papers  (2024-08-01T05:46:57Z)
• FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models [85.16273912625022]
  We introduce FaceTalk, a novel generative approach designed for synthesizing high-fidelity 3D motion sequences of talking human heads from audio signal. To the best of our knowledge, this is the first work to propose a generative approach for realistic and high-quality motion synthesis of human heads.
  arXiv  Detail & Related papers  (2023-12-13T19:01:07Z)
• GMTalker: Gaussian Mixture-based Audio-Driven Emotional Talking Video Portraits [60.05683966405544]
  We present GMTalker, a Gaussian mixture-based emotional talking portraits generation framework.<n>Specifically, we propose a continuous and disentangled latent space, achieving more flexible emotion manipulation.<n>We also introduce a normalizing flow-based motion generator pretrained on a large dataset to generate diverse head poses, blinks, and eyeball movements.
  arXiv  Detail & Related papers  (2023-12-12T19:03:04Z)
• EmotionGesture: Audio-Driven Diverse Emotional Co-Speech 3D Gesture Generation [24.547098909937034]
  EmotionGesture is a novel framework for vivid and diverse emotional co-speech 3D gestures from audio. Our framework outperforms the state-of-the-art, achieving vivid and diverse emotional co-speech 3D gestures.
  arXiv  Detail & Related papers  (2023-05-30T09:47:29Z)
• Controllable Radiance Fields for Dynamic Face Synthesis [125.48602100893845]
  We study how to explicitly control generative model synthesis of face dynamics exhibiting non-rigid motion. Controllable Radiance Field (CoRF) On head image/video data we show that CoRFs are 3D-aware while enabling editing of identity, viewing directions, and motion.
  arXiv  Detail & Related papers  (2022-10-11T23:17:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.