CETalk: Continuous Valence-Arousal Control for Audio-Driven 3D Talking Head Generation

• URL: http://arxiv.org/abs/2608.15110v1
• Date: Sat, 15 Aug 2026 08:22:50 GMT
• Title: CETalk: Continuous Valence-Arousal Control for Audio-Driven 3D Talking Head Generation
• Abstract summary: We propose CETalk, an audio-driven 3D facial animation framework conditioned on Valence--Arousal representations for fine-grained emotion control.<n>We show that CETalk outperforms state-of-the-art methods in both lip-sync accuracy and emotional expressiveness.
• Score: 18.841894940869157
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Emotional 3D talking head generation aims to synthesize expressive facial animations with accurate lip synchronization. However, existing methods often rely on discrete emotion categories, which fail to capture the continuous evolution of affect. They also overlook the temporal frequency mismatch between audio articulation and emotional expression. In this paper, we propose CETalk, an audio-driven 3D facial animation framework conditioned on continuous Valence--Arousal (VA) representations for fine-grained emotion control. CETalk predicts a sequence of FLAME parameters through three key components: a Dynamic Emotion Modulation Module that adaptively scales emotional intensity using audio-derived cues; a Multi-Scale Temporal Modeling mechanism that employs parallel branches to decouple high-frequency articulatory movements from low-frequency emotional dynamics; and a Dynamic Fusion Mechanism that integrates these multi-scale features via an adaptive gating network. To support training and evaluation, we construct 3D-VA-MEAD, a large-scale dataset with automatically estimated VA annotations and reconstructed 3D facial motions. Extensive experiments demonstrate that CETalk outperforms state-of-the-art methods in both lip-sync accuracy and emotional expressiveness, while enabling smooth and controllable emotion transitions.

Related papers

• SEDTalker: Emotion-Aware 3D Facial Animation Using Frame-Level Speech Emotion Diarization [14.632533340477591]
  SEDTalker is an emotion-aware framework for speech-driven 3D facial animation.<n>We use frame-level speech emotion diarization to achieve fine-grained expressive control.
  arXiv  Detail & Related papers  (2026-04-14T22:55:09Z)
• 3DXTalker: Unifying Identity, Lip Sync, Emotion, and Spatial Dynamics in Expressive 3D Talking Avatars [46.32463788372058]
  3DXTalker is an expressive 3D talking avatar through data-curated identity modeling, audio-rich representations, and spatial dynamics controllability.<n>We introduce frame-wise amplitude and emotional cues beyond standard speech embeddings, ensuring superior lip synchronization and nuanced expression modulation.
  arXiv  Detail & Related papers  (2026-02-11T04:31:13Z)
• EditEmoTalk: Controllable Speech-Driven 3D Facial Animation with Continuous Expression Editing [24.15552429255594]
  Speech-driven 3D facial animation aims to generate realistic and expressive facial motions directly from audio.<n>We present EditEmoTalk, a controllable speech-driven 3D facial animation framework with continuous emotion editing.
  arXiv  Detail & Related papers  (2026-01-15T02:21:22Z)
• EmoCAST: Emotional Talking Portrait via Emotive Text Description [56.42674612728354]
  EmoCAST is a diffusion-based framework for precise text-driven emotional synthesis.<n>In appearance modeling, emotional prompts are integrated through a text-guided decoupled emotive module.<n>EmoCAST achieves state-of-the-art performance in generating realistic, emotionally expressive, and audio-synchronized talking-head videos.
  arXiv  Detail & Related papers  (2025-08-28T10:02:06Z)
• HM-Talker: Hybrid Motion Modeling for High-Fidelity Talking Head Synthesis [90.74616208952791]
  HM-Talker is a novel framework for generating high-fidelity, temporally coherent talking heads.<n>Explicit cues use Action Units (AUs), anatomically defined facial muscle movements, alongside implicit features to minimize phoneme-viseme misalignment.
  arXiv  Detail & Related papers  (2025-08-14T12:01:52Z)
• MEDTalk: Multimodal Controlled 3D Facial Animation with Dynamic Emotions by Disentangled Embedding [48.54455964043634]
  MEDTalk is a novel framework for fine-grained and dynamic emotional talking head generation.<n>We integrate audio and speech text, predicting frame-wise intensity variations and dynamically adjusting static emotion features to generate realistic emotional expressions.<n>Our generated results can be conveniently integrated into the industrial production pipeline.
  arXiv  Detail & Related papers  (2025-07-08T15:14:27Z)
• EmoDiffusion: Enhancing Emotional 3D Facial Animation with Latent Diffusion Models [66.67979602235015]
  EmoDiffusion is a novel approach that disentangles different emotions in speech to generate rich 3D emotional facial expressions.<n>We capture facial expressions under the guidance of animation experts using LiveLinkFace on an iPhone.
  arXiv  Detail & Related papers  (2025-03-14T02:54:22Z)
• Emotional Speech-Driven Animation with Content-Emotion Disentanglement [51.34635009347183]
  We propose EMOTE, which generates 3D talking-head avatars that maintain lip-sync from speech while enabling explicit control over the expression of emotion. EmOTE produces speech-driven facial animations with better lip-sync than state-of-the-art methods trained on the same data.
  arXiv  Detail & Related papers  (2023-06-15T09:31:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.