EditEmoTalk: Controllable Speech-Driven 3D Facial Animation with Continuous Expression Editing

• URL: http://arxiv.org/abs/2601.10000v1
• Date: Thu, 15 Jan 2026 02:21:22 GMT
• Title: EditEmoTalk: Controllable Speech-Driven 3D Facial Animation with Continuous Expression Editing
• Authors: Diqiong Jiang, Kai Zhu, Dan Song, Jian Chang, Chenglizhao Chen, Zhenyu Wu,
• Abstract summary: Speech-driven 3D facial animation aims to generate realistic and expressive facial motions directly from audio.<n>We present EditEmoTalk, a controllable speech-driven 3D facial animation framework with continuous emotion editing.
• Score: 24.15552429255594
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Speech-driven 3D facial animation aims to generate realistic and expressive facial motions directly from audio. While recent methods achieve high-quality lip synchronization, they often rely on discrete emotion categories, limiting continuous and fine-grained emotional control. We present EditEmoTalk, a controllable speech-driven 3D facial animation framework with continuous emotion editing. The key idea is a boundary-aware semantic embedding that learns the normal directions of inter-emotion decision boundaries, enabling a continuous expression manifold for smooth emotion manipulation. Moreover, we introduce an emotional consistency loss that enforces semantic alignment between the generated motion dynamics and the target emotion embedding through a mapping network, ensuring faithful emotional expression. Extensive experiments demonstrate that EditEmoTalk achieves superior controllability, expressiveness, and generalization while maintaining accurate lip synchronization. Code and pretrained models will be released.

Related papers

• Think2Sing: Orchestrating Structured Motion Subtitles for Singing-Driven 3D Head Animation [69.50178144839275]
  Singing involves richer emotional nuance, dynamic prosody, and lyric-based semantics.<n>Existing speech-driven approaches often produce oversimplified, emotionally flat, and semantically inconsistent results.<n>Think2Sing generates semantically coherent and temporally consistent 3D head animations conditioned on both lyrics and acoustics.
  arXiv  Detail & Related papers  (2025-09-02T12:59:27Z)
• EmoCAST: Emotional Talking Portrait via Emotive Text Description [56.42674612728354]
  EmoCAST is a diffusion-based framework for precise text-driven emotional synthesis.<n>In appearance modeling, emotional prompts are integrated through a text-guided decoupled emotive module.<n>EmoCAST achieves state-of-the-art performance in generating realistic, emotionally expressive, and audio-synchronized talking-head videos.
  arXiv  Detail & Related papers  (2025-08-28T10:02:06Z)
• MEDTalk: Multimodal Controlled 3D Facial Animation with Dynamic Emotions by Disentangled Embedding [48.54455964043634]
  MEDTalk is a novel framework for fine-grained and dynamic emotional talking head generation.<n>We integrate audio and speech text, predicting frame-wise intensity variations and dynamically adjusting static emotion features to generate realistic emotional expressions.<n>Our generated results can be conveniently integrated into the industrial production pipeline.
  arXiv  Detail & Related papers  (2025-07-08T15:14:27Z)
• EmoDiffusion: Enhancing Emotional 3D Facial Animation with Latent Diffusion Models [66.67979602235015]
  EmoDiffusion is a novel approach that disentangles different emotions in speech to generate rich 3D emotional facial expressions.<n>We capture facial expressions under the guidance of animation experts using LiveLinkFace on an iPhone.
  arXiv  Detail & Related papers  (2025-03-14T02:54:22Z)
• DEEPTalk: Dynamic Emotion Embedding for Probabilistic Speech-Driven 3D Face Animation [14.07086606183356]
  Speech-driven 3D facial animation has garnered lots of attention thanks to its broad range of applications.<n>Current methods fail to capture the nuanced emotional undertones conveyed through speech and produce monotonous facial motion.<n>We introduce DEEPTalk, a novel approach that generates diverse and emotionally rich 3D facial expressions directly from speech inputs.
  arXiv  Detail & Related papers  (2024-08-12T08:56:49Z)
• DREAM-Talk: Diffusion-based Realistic Emotional Audio-driven Method for Single Image Talking Face Generation [75.90730434449874]
  We introduce DREAM-Talk, a two-stage diffusion-based audio-driven framework, tailored for generating diverse expressions and accurate lip-sync concurrently. Given the strong correlation between lip motion and audio, we then refine the dynamics with enhanced lip-sync accuracy using audio features and emotion style. Both quantitatively and qualitatively, DREAM-Talk outperforms state-of-the-art methods in terms of expressiveness, lip-sync accuracy and perceptual quality.
  arXiv  Detail & Related papers  (2023-12-21T05:03:18Z)
• Emotional Speech-Driven Animation with Content-Emotion Disentanglement [51.34635009347183]
  We propose EMOTE, which generates 3D talking-head avatars that maintain lip-sync from speech while enabling explicit control over the expression of emotion. EmOTE produces speech-driven facial animations with better lip-sync than state-of-the-art methods trained on the same data.
  arXiv  Detail & Related papers  (2023-06-15T09:31:31Z)
• Expressive Speech-driven Facial Animation with controllable emotions [12.201573788014622]
  This paper presents a novel deep learning-based approach for expressive facial animation generation from speech. It can exhibit wide-spectrum facial expressions with controllable emotion type and intensity. It enables emotion-controllable facial animation, where the target expression can be continuously adjusted.
  arXiv  Detail & Related papers  (2023-01-05T11:17:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.