ECHO: Towards Emotionally Appropriate and Contextually Aware Interactive Head Generation

• URL: http://arxiv.org/abs/2603.17427v1
• Date: Wed, 18 Mar 2026 07:07:01 GMT
• Title: ECHO: Towards Emotionally Appropriate and Contextually Aware Interactive Head Generation
• Abstract summary: Interactive Head Generation (IHG) aims to synthesize lifelike avatar head video emulating such capabilities.<n>ECHO is a novel IHG framework comprising two key components: a Long-range Contextual Understanding (LCU) component and a block-wise Spatial-aware Decoupled Cross-attention Modulation (SDCM) module.
• Score: 37.457960520410246
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In natural face-to-face interaction, participants seamlessly alternate between speaking and listening, producing facial behaviors (FBs) that are finely informed by long-range context and naturally exhibit contextual appropriateness and emotional rationality. Interactive Head Generation (IHG) aims to synthesize lifelike avatar head video emulating such capabilities. Existing IHG methods typically condition on dual-track signals (i.e., human user's behaviors and pre-defined audio for avatar) within a short temporal window, jointly driving generation of avatar's audio-aligned lip articulation and non-verbal FBs. However, two main challenges persist in these methods: (i) the reliance on short-clip behavioral cues without long-range contextual modeling leads them to produce facial behaviors lacking contextual appropriateness; and (ii) the entangled, role-agnostic fusion of dual-track signals empirically introduces cross-signal interference, potentially compromising lip-region synchronization during speaking. To this end, we propose ECHO, a novel IHG framework comprising two key components: a Long-range Contextual Understanding (LCU) component that facilitates contextual understanding of both behavior-grounded dynamics and linguistic-driven affective semantics to promote contextual appropriateness and emotional rationality of synthesized avatar FBs; and a block-wise Spatial-aware Decoupled Cross-attention Modulation (SDCM) module, that preserves self-audio-driven lip articulation while adaptively integrating user contextual behavioral cues for non-lip facial regions, complemented by our designed two-stage training paradigm, to jointly enhance lip synchronization and visual fidelity. Extensive experiments demonstrate the effectiveness of proposed components and ECHO's superior IHG performance.

Related papers

• Adaptive Physical-Facial Representation Fusion via Subject-Invariant Cross-Modal Prompt Tuning for Video-Based Emotion Recognition [14.850489141041017]
  We propose a cross-modal cross-tuning prompt-tuning method for video-based emotion recognition.<n>The proposed method consistently outperforms strong baselines in both recognition accuracy and generalization ability.
  arXiv  Detail & Related papers  (2026-05-07T05:33:31Z)
• EmbodiedHead: Real-Time Listening and Speaking Avatar for Conversational Agents [5.090098283091398]
  EmHead is a speech-driven talking-head framework that equips LLMs with real-time visual avatars for conversation.<n>A practical embodied avatar must achieve real-time generation, unified listening-speaking behavior, and high rendered visual quality simultaneously.
  arXiv  Detail & Related papers  (2026-04-19T02:43:00Z)
• Multimodal Emotion Recognition via Bi-directional Cross-Attention and Temporal Modeling [2.8037951156321377]
  We propose a multimodal emotion recognition framework for the Expression task in the 10th Affective Behavior Analysis in-the-wild (ABAW) Challenge.<n>Our framework builds on large-scale pre-trained models for visual and audio representation learning and integrates them in a unified multimodal architecture.<n> Experimental results on the ABAW 10th EXPR benchmark demonstrate the effectiveness of the proposed framework.
  arXiv  Detail & Related papers  (2026-03-12T14:20:29Z)
• Facial Expression Generation Aligned with Human Preference for Natural Dyadic Interaction [53.12445980832305]
  We propose a facial expression generation method aligned with human preference.<n>A key to our method is framing the generation of identity-independent facial expressions as an action learning process.<n> Experiments on two benchmarks demonstrate that our method effectively aligns facial expressions with human preference.
  arXiv  Detail & Related papers  (2026-03-07T08:03:21Z)
• MIBURI: Towards Expressive Interactive Gesture Synthesis [62.45332399212876]
  Embodied Conversational Agents (ECAs) aim to emulate human face-to-face interaction through speech, gestures, and facial expressions.<n>Existing solutions for ECAs produce rigid, low-diversity motions that are unsuitable for human-like interaction.<n>We present MIBURI, the first online, causal framework for generating expressive full-body gestures and facial expressions synchronized with real-time spoken dialogue.
  arXiv  Detail & Related papers  (2026-03-03T18:59:51Z)
• HM-Talker: Hybrid Motion Modeling for High-Fidelity Talking Head Synthesis [90.74616208952791]
  HM-Talker is a novel framework for generating high-fidelity, temporally coherent talking heads.<n>Explicit cues use Action Units (AUs), anatomically defined facial muscle movements, alongside implicit features to minimize phoneme-viseme misalignment.
  arXiv  Detail & Related papers  (2025-08-14T12:01:52Z)
• Text2Lip: Progressive Lip-Synced Talking Face Generation from Text via Viseme-Guided Rendering [53.2204901422631]
  Text2Lip is a viseme-centric framework that constructs an interpretable phonetic-visual bridge.<n>We show that Text2Lip outperforms existing approaches in semantic fidelity, visual realism, and modality robustness.
  arXiv  Detail & Related papers  (2025-08-04T12:50:22Z)
• ARIG: Autoregressive Interactive Head Generation for Real-time Conversations [15.886402427095515]
  Face-to-face communication, as a common human activity, motivates the research on interactive head generation.<n>Previous clip-wise generation paradigm or explicit listener/speaker generator-switching methods have limitations in future signal acquisition.<n>We propose an autoregressive (AR) based frame-wise framework called ARIG to realize the real-time generation with better interaction realism.
  arXiv  Detail & Related papers  (2025-07-01T06:38:14Z)
• RealTalk: Real-time and Realistic Audio-driven Face Generation with 3D Facial Prior-guided Identity Alignment Network [48.95833484103569]
  RealTalk is an audio-to-expression transformer and a high-fidelity expression-to-face framework. In the first component, we consider both identity and intra-personal variation features related to speaking lip movements. In the second component, we design a lightweight facial identity alignment (FIA) module. This novel design allows us to generate fine details in real-time, without depending on sophisticated and inefficient feature alignment modules.
  arXiv  Detail & Related papers  (2024-06-26T12:09:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.