VividFace: Real-Time and Realistic Facial Expression Shadowing for Humanoid Robots

• URL: http://arxiv.org/abs/2602.07506v2
• Date: Sat, 14 Feb 2026 10:07:17 GMT
• Title: VividFace: Real-Time and Realistic Facial Expression Shadowing for Humanoid Robots
• Abstract summary: We present VividFace, a real-time and realistic facial expression shadowing system for humanoid robots.<n>VividFace produces vivid humanoid faces by mimicking human facial expressions within 0.05 seconds.
• Score: 37.61907490582774
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humanoid facial expression shadowing enables robots to realistically imitate human facial expressions in real time, which is critical for lifelike, facially expressive humanoid robots and affective human-robot interaction. Existing progress in humanoid facial expression imitation remains limited, often failing to achieve either real-time performance or realistic expressiveness due to offline video-based inference designs and insufficient ability to capture and transfer subtle expression details. To address these limitations, we present VividFace, a real-time and realistic facial expression shadowing system for humanoid robots. An optimized imitation framework X2CNet++ enhances expressiveness by fine-tuning the human-to-humanoid facial motion transfer module and introducing a feature-adaptation training strategy for better alignment across different image sources. Real-time shadowing is further enabled by a video-stream-compatible inference pipeline and a streamlined workflow based on asynchronous I/O for efficient communication across devices. VividFace produces vivid humanoid faces by mimicking human facial expressions within 0.05 seconds, while generalizing across diverse facial configurations. Extensive real-world demonstrations validate its practical utility. Videos are available at: https://lipzh5.github.io/VividFace/.

Related papers

• Morphology-Independent Facial Expression Imitation for Human-Face Robots [31.41201394665696]
  We propose a morphology-independent expression imitation method that decouples expressions from facial morphology.<n>Our method enables the human-face robot to reproduce a wide range of human-like expressions effectively.
  arXiv  Detail & Related papers  (2026-03-07T06:57:52Z)
• Audio Driven Real-Time Facial Animation for Social Telepresence [65.66220599734338]
  We present an audio-driven real-time system for animating photorealistic 3D facial avatars with minimal latency.<n>Central to our approach is an encoder model that transforms audio signals into latent facial expression sequences in real time.<n>We capture the rich spectrum of facial expressions necessary for natural communication while achieving real-time performance.
  arXiv  Detail & Related papers  (2025-10-01T17:57:05Z)
• X2C: A Dataset Featuring Nuanced Facial Expressions for Realistic Humanoid Imitation [34.960259285861206]
  The ability to imitate realistic facial expressions is essential for humanoid robots engaged in affective human-robot communication.<n>We introduce X2C, a dataset featuring nuanced facial expressions for realistic humanoid imitation.<n>X2CNet, a novel human-to-humanoid facial expression imitation framework, learns the correspondence between nuanced humanoid expressions and their underlying control values from X2C.
  arXiv  Detail & Related papers  (2025-05-16T11:48:19Z)
• EmoDiffusion: Enhancing Emotional 3D Facial Animation with Latent Diffusion Models [66.67979602235015]
  EmoDiffusion is a novel approach that disentangles different emotions in speech to generate rich 3D emotional facial expressions.<n>We capture facial expressions under the guidance of animation experts using LiveLinkFace on an iPhone.
  arXiv  Detail & Related papers  (2025-03-14T02:54:22Z)
• X-Dyna: Expressive Dynamic Human Image Animation [49.896933584815926]
  X-Dyna is a zero-shot, diffusion-based pipeline for animating a single human image.<n>It generates realistic, context-aware dynamics for both the subject and the surrounding environment.
  arXiv  Detail & Related papers  (2025-01-17T08:10:53Z)
• CSTalk: Correlation Supervised Speech-driven 3D Emotional Facial Animation Generation [13.27632316528572]
  Speech-driven 3D facial animation technology has been developed for years, but its practical application still lacks expectations. Main challenges lie in data limitations, lip alignment, and the naturalness of facial expressions. This paper proposes a method called CSTalk that models the correlations among different regions of facial movements and supervises the training of the generative model to generate realistic expressions.
  arXiv  Detail & Related papers  (2024-04-29T11:19:15Z)
• Driving Animatronic Robot Facial Expression From Speech [7.8799497614708605]
  This paper introduces a novel, skinning-centric approach to drive animatronic robot facial expressions from speech input. The proposed approach employs linear skinning (LBS) as a unifying representation, guiding innovations in both embodiment design and motion synthesis. This approach demonstrates the capability to produce highly realistic facial expressions on an animatronic face in real-time at over 4000 fps on a single Nvidia GTX 4090.
  arXiv  Detail & Related papers  (2024-03-19T12:11:57Z)
• RealDex: Towards Human-like Grasping for Robotic Dexterous Hand [64.33746404551343]
  We introduce RealDex, a pioneering dataset capturing authentic dexterous hand grasping motions infused with human behavioral patterns.<n>RealDex holds immense promise in advancing humanoid robot for automated perception, cognition, and manipulation in real-world scenarios.
  arXiv  Detail & Related papers  (2024-02-21T14:59:46Z)
• Audio-Driven Talking Face Generation with Diverse yet Realistic Facial Animations [61.65012981435094]
  DIRFA is a novel method that can generate talking faces with diverse yet realistic facial animations from the same driving audio. To accommodate fair variation of plausible facial animations for the same audio, we design a transformer-based probabilistic mapping network. We show that DIRFA can generate talking faces with realistic facial animations effectively.
  arXiv  Detail & Related papers  (2023-04-18T12:36:15Z)
• Emotionally Enhanced Talking Face Generation [52.07451348895041]
  We build a talking face generation framework conditioned on a categorical emotion to generate videos with appropriate expressions. We show that our model can adapt to arbitrary identities, emotions, and languages. Our proposed framework is equipped with a user-friendly web interface with a real-time experience for talking face generation with emotions.
  arXiv  Detail & Related papers  (2023-03-21T02:33:27Z)
• CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior [27.989344587876964]
  Speech-driven 3D facial animation has been widely studied, yet there is still a gap to achieving realism and vividness. We propose to cast speech-driven facial animation as a code query task in a finite proxy space of the learned codebook. We demonstrate that our approach outperforms current state-of-the-art methods both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2023-01-06T05:04:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.