Real-Time Synchronized Interaction Framework for Emotion-Aware Humanoid Robots

• URL: http://arxiv.org/abs/2601.17287v1
• Date: Sat, 24 Jan 2026 03:53:09 GMT
• Title: Real-Time Synchronized Interaction Framework for Emotion-Aware Humanoid Robots
• Abstract summary: We present a real-time framework for NAO robots that synchronizes speech prosody with full-body gestures.<n>By enabling seamless sensorimotor coordination, this framework advances the deployment of context-aware social robots.
• Score: 0.5156484100374058
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As humanoid robots increasingly introduced into social scene, achieving emotionally synchronized multimodal interaction remains a significant challenges. To facilitate the further adoption and integration of humanoid robots into service roles, we present a real-time framework for NAO robots that synchronizes speech prosody with full-body gestures through three key innovations: (1) A dual-channel emotion engine where large language model (LLM) simultaneously generates context-aware text responses and biomechanically feasible motion descriptors, constrained by a structured joint movement library; (2) Duration-aware dynamic time warping for precise temporal alignment of speech output and kinematic motion keyframes; (3) Closed-loop feasibility verification ensuring gestures adhere to NAO's physical joint limits through real-time adaptation. Evaluations show 21% higher emotional alignment compared to rule-based systems, achieved by coordinating vocal pitch (arousal-driven) with upper-limb kinematics while maintaining lower-body stability. By enabling seamless sensorimotor coordination, this framework advances the deployment of context-aware social robots in dynamic applications such as personalized healthcare, interactive education, and responsive customer service platforms.

Related papers

• Automated Synthesis of Facial Mechanisms for Conversational Animatronic Robots [56.50769770726642]
  State-of-the-art animatronic faces are typically tailored systems, making large-scale personalization slow and costly.<n>We propose a parametric, linkage-driven mechanical face template whose topology and actuator layout are explicitly parameterized to support systematic scaling.<n>We develop a dual-identity conversational facial motion synthesis framework that jointly models speaking and listening behaviors from audio.
  arXiv  Detail & Related papers  (2026-07-13T15:22:46Z)
• PhysDrift: Bridging the Embodiment Gap in Humanoid Co-Speech Motion Generation [31.168201029670865]
  We introduce PhysDrift, an embodiment-aware co-speech motion framework that predicts humanoid joint trajectories from speech without relying on human-body representations.<n>Experiments and real-world humanoid deployment demonstrate that embodiment-aware robot-native generation substantially improves speech-motion alignment, physical plausibility, motion smoothness, inference efficiency, and real-time interaction capability.
  arXiv  Detail & Related papers  (2026-06-18T08:31:58Z)
• WaveSync: Constrained Wavefront Optimization for Synchronized Co-Speech Gestures in Humanoid Robots [2.0424038686449006]
  Expressive co-speech gestures are crucial for natural human-robot interaction, but generating them on physical humanoid robots is difficult.<n>We present textbfWaveSync, a hybrid framework in which a Large Language Model decomposes dialogue responses into structured semantic schemas.<n>We show that our method achieves high synchronization accuracy and outperforms three baselines in both objective and subjective evaluations.
  arXiv  Detail & Related papers  (2026-06-15T11:47:30Z)
• HO-Flow: Generalizable Hand-Object Interaction Generation with Latent Flow Matching [113.81911881001905]
  HO-Flow is a framework for synthesizing realistic hand-object motion sequences from texts and canoncial 3D objects.<n>It first employs an interaction-aware variational autoencoder to encode sequences of hand and object motions into a unified latent manifold.<n>It then leverages a masked flow matching model that combines auto-regressive temporal reasoning with continuous latent generation.
  arXiv  Detail & Related papers  (2026-04-12T22:06:11Z)
• MIBURI: Towards Expressive Interactive Gesture Synthesis [62.45332399212876]
  Embodied Conversational Agents (ECAs) aim to emulate human face-to-face interaction through speech, gestures, and facial expressions.<n>Existing solutions for ECAs produce rigid, low-diversity motions that are unsuitable for human-like interaction.<n>We present MIBURI, the first online, causal framework for generating expressive full-body gestures and facial expressions synchronized with real-time spoken dialogue.
  arXiv  Detail & Related papers  (2026-03-03T18:59:51Z)
• Rhythm: Learning Interactive Whole-Body Control for Dual Humanoids [25.74949263474117]
  We introduce Rhythm, the first unified framework enabling real-world deployment of dual-humanoid systems for complex, physically plausible interactions.<n>Our framework integrates three core components: (1) an Interaction-Aware Motion Retargeting (IAMR) module that generates feasible humanoid interaction references from human data; (2) an Interaction-Guided Reinforcement Learning (IGRL) policy that masters coupled dynamics via graph-based rewards; and (3) a real-world deployment system that enables robust transfer of dual-humanoid interaction.
  arXiv  Detail & Related papers  (2026-03-03T11:04:56Z)
• Physical Human-Robot Interaction for Grasping in Augmented Reality via Rigid-Soft Robot Synergy [43.680646123465316]
  Hybrid rigid-soft robots combine the precision of rigid manipulators with the compliance and adaptability of soft arms.<n>We present a novel augmented reality (AR)-based physical human-robot interaction framework that enables direct teleoperation of a hybrid rigid-soft robot.
  arXiv  Detail & Related papers  (2026-02-19T06:56:47Z)
• Interact2Ar: Full-Body Human-Human Interaction Generation via Autoregressive Diffusion Models [80.28579390566298]
  We introduce Interact2Ar, a text-conditioned autoregressive diffusion model for generating full-body, human-human interactions.<n>Hand kinematics are incorporated through dedicated parallel branches, enabling high-fidelity full-body generation.<n>Our model enables a series of downstream applications, including temporal motion composition, real-time adaptation to disturbances, and extension beyond dyadic to multi-person scenarios.
  arXiv  Detail & Related papers  (2025-12-22T18:59:50Z)
• Semantic Co-Speech Gesture Synthesis and Real-Time Control for Humanoid Robots [5.531678625546847]
  We present an innovative end-to-end framework for synthesizing semantically meaningful co-speech gestures and deploying them in real-time on a humanoid robot.<n>Our core innovation lies in the meticulous integration of a semantics-aware gesture synthesis module.<n>We demonstrate that our combined system produces semantically appropriate and rhythmically coherent gestures.
  arXiv  Detail & Related papers  (2025-12-19T02:55:10Z)
• Towards Immersive Human-X Interaction: A Real-Time Framework for Physically Plausible Motion Synthesis [51.95817740348585]
  Human-X is a novel framework designed to enable immersive and physically plausible human interactions across diverse entities.<n>Our method jointly predicts actions and reactions in real-time using an auto-regressive reaction diffusion planner.<n>Our framework is validated in real-world applications, including virtual reality interface for human-robot interaction.
  arXiv  Detail & Related papers  (2025-08-04T06:35:48Z)
• Robot Interaction Behavior Generation based on Social Motion Forecasting for Human-Robot Interaction [9.806227900768926]
  We propose to model social motion forecasting in a shared human-robot representation space. ECHO operates in the aforementioned shared space to predict the future motions of the agents encountered in social scenarios. We evaluate our model in multi-person and human-robot motion forecasting tasks and obtain state-of-the-art performance by a large margin.
  arXiv  Detail & Related papers  (2024-02-07T11:37:14Z)
• ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactions [66.87211993793807]
  We present ReMoS, a denoising diffusion based model that synthesizes full body motion of a person in two person interaction scenario. We demonstrate ReMoS across challenging two person scenarios such as pair dancing, Ninjutsu, kickboxing, and acrobatics. We also contribute the ReMoCap dataset for two person interactions containing full body and finger motions.
  arXiv  Detail & Related papers  (2023-11-28T18:59:52Z)
• Skeleton2Humanoid: Animating Simulated Characters for Physically-plausible Motion In-betweening [59.88594294676711]
  Modern deep learning based motion synthesis approaches barely consider the physical plausibility of synthesized motions. We propose a system Skeleton2Humanoid'' which performs physics-oriented motion correction at test time. Experiments on the challenging LaFAN1 dataset show our system can outperform prior methods significantly in terms of both physical plausibility and accuracy.
  arXiv  Detail & Related papers  (2022-10-09T16:15:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.