PhysDrift: Bridging the Embodiment Gap in Humanoid Co-Speech Motion Generation

• URL: http://arxiv.org/abs/2606.19935v1
• Date: Thu, 18 Jun 2026 08:31:58 GMT
• Title: PhysDrift: Bridging the Embodiment Gap in Humanoid Co-Speech Motion Generation
• Abstract summary: We introduce PhysDrift, an embodiment-aware co-speech motion framework that predicts humanoid joint trajectories from speech without relying on human-body representations.<n>Experiments and real-world humanoid deployment demonstrate that embodiment-aware robot-native generation substantially improves speech-motion alignment, physical plausibility, motion smoothness, inference efficiency, and real-time interaction capability.
• Score: 31.168201029670865
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humanoid robots require co-speech motions that are not only expressive and speech-aligned, but also physically executable under embodiment constraints. Existing co-speech generation pipelines are predominantly human-centric: motions are first generated in human-body representations such as SMPL-X and subsequently retargeted to humanoid robots. In this work, we identify a fundamental embodiment gap in this paradigm, where the mismatch between human motion manifolds and humanoid embodiment constraints disrupts embodiment consistency during motion transfer and physical execution. Through extensive analysis, we show that although retargeting can preserve coarse motion semantics, it significantly compresses motion diversity and weakens prosody-motion synchronization, limiting expressive humanoid behaviors. To address this problem, we first propose IK-EER, a prosody-preserving humanoid motion curation framework that jointly optimizes kinematic feasibility and speech-motion temporal alignment during retargeting. Building upon the curated robot-native motion dataset, we further introduce PhysDrift, an embodiment-aware co-speech motion generation framework that directly predicts executable humanoid joint trajectories from speech without relying on intermediate human-body representations. Unlike conventional human-centric pipelines, PhysDrift maintains embodiment consistency throughout both training and inference while incorporating physical regularization to stabilize robot motion dynamics. Extensive experiments and real-world humanoid deployment demonstrate that embodiment-aware robot-native generation substantially improves speech-motion alignment, physical plausibility, motion smoothness, inference efficiency, and real-time interaction capability.

Related papers

• Human2Humanoid: Physics-Aware Cross-Morphology Motion Retargeting for Humanoid Robots [9.096721498763717]
  Retargeting human motion to humanoid robots is critical for teleoperation, imitation learning and human-robot interaction.<n>This paper presents Human2Human, an unsupervised motion framework that transfers human motions to humanoid robot behaviors with high fidelity.<n> Experimental results show that the proposed method successfully retargets human motion to the Unitree G1 humanoid robot without paired data.
  arXiv  Detail & Related papers  (2026-06-02T10:53:06Z)
• SocialMirror: Reconstructing 3D Human Interaction Behaviors from Monocular Videos with Semantic and Geometric Guidance [49.69016078147708]
  Accurately reconstructing human behavior in close-interaction scenarios is crucial for enabling realistic virtual interactions in augmented reality.<n>We propose SocialMirror, a diffusion-based framework that integrates semantic and geometric cues to effectively address these issues.<n>SocialMirror achieves state-of-the-art performance in reconstructing interactive human meshes, demonstrating strong generalization across unseen datasets and in-the-wild scenarios.
  arXiv  Detail & Related papers  (2026-04-15T07:41:52Z)
• RoboForge: Physically Optimized Text-guided Whole-Body Locomotion for Humanoids [20.796118584632904]
  We present a unified latent-driven framework that bridges natural language and whole-body locomotion.<n>Our framework provides a practical path toward deployable text-guided humanoid intelligence.
  arXiv  Detail & Related papers  (2026-03-18T17:02:56Z)
• PhyGile: Physics-Prefix Guided Motion Generation for Agile General Humanoid Motion Tracking [23.96905373874686]
  We present PhyGile, a unified framework that closes the loop between robot-native motion generation and inference.<n>We train the GMT controller with a curriculum-based mixture-of-experts scheme, followed by post-training on unlabeled motion data to improve robustness over large-scale robot motions.<n>PhyGile expands the frontier of text-driven humanoid control, enabling stable tracking of agile, highly difficult whole-body motions.
  arXiv  Detail & Related papers  (2026-03-13T06:43:44Z)
• AdaMorph: Unified Motion Retargeting via Embodiment-Aware Adaptive Transformers [49.796014934503184]
  We present a unified neural framework that enables a single model to adapt human motion to diverse robot morphologies.<n> Experimental results on 12 distinct humanoid robots demonstrate that AdaMorph effectively control across heterogeneous topologies.
  arXiv  Detail & Related papers  (2026-01-12T07:39:38Z)
• Semantic Co-Speech Gesture Synthesis and Real-Time Control for Humanoid Robots [5.531678625546847]
  We present an innovative end-to-end framework for synthesizing semantically meaningful co-speech gestures and deploying them in real-time on a humanoid robot.<n>Our core innovation lies in the meticulous integration of a semantics-aware gesture synthesis module.<n>We demonstrate that our combined system produces semantically appropriate and rhythmically coherent gestures.
  arXiv  Detail & Related papers  (2025-12-19T02:55:10Z)
• Commanding Humanoid by Free-form Language: A Large Language Action Model with Unified Motion Vocabulary [59.98573566227095]
  We introduce Humanoid-LLA, a Large Language Action Model that maps expressive language commands to physically executable whole-body actions for humanoid robots.<n>Our approach integrates three core components: a unified motion vocabulary that aligns human and humanoid motion primitives into a shared discrete space; a vocabulary-directed controller distilled from a privileged policy to ensure physical feasibility; and a physics-informed fine-tuning stage using reinforcement learning with dynamics-aware rewards to enhance robustness and stability.
  arXiv  Detail & Related papers  (2025-11-28T08:11:24Z)
• From Language to Locomotion: Retargeting-free Humanoid Control via Motion Latent Guidance [55.31807046722006]
  Existing language-guided humanoid pipelines are cumbersome and untrustworthy.<n>We present RoboGhost, a language-free framework that conditions humanoid policies on language-grounded motion latents.<n>We show that RoboGhost substantially reduces deployment latency, improves success rates and tracking precision, and produces smooth, semantically aligned humanoids.
  arXiv  Detail & Related papers  (2025-10-16T17:57:47Z)
• Towards Immersive Human-X Interaction: A Real-Time Framework for Physically Plausible Motion Synthesis [51.95817740348585]
  Human-X is a novel framework designed to enable immersive and physically plausible human interactions across diverse entities.<n>Our method jointly predicts actions and reactions in real-time using an auto-regressive reaction diffusion planner.<n>Our framework is validated in real-world applications, including virtual reality interface for human-robot interaction.
  arXiv  Detail & Related papers  (2025-08-04T06:35:48Z)
• Skeleton2Humanoid: Animating Simulated Characters for Physically-plausible Motion In-betweening [59.88594294676711]
  Modern deep learning based motion synthesis approaches barely consider the physical plausibility of synthesized motions. We propose a system Skeleton2Humanoid'' which performs physics-oriented motion correction at test time. Experiments on the challenging LaFAN1 dataset show our system can outperform prior methods significantly in terms of both physical plausibility and accuracy.
  arXiv  Detail & Related papers  (2022-10-09T16:15:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.