Human-as-Humanoid: Enabling Zero-Shot Humanoid Learning from Ego-Exo Human Videos with Human-Aligned Embodiments
- URL: http://arxiv.org/abs/2606.32009v1
- Date: Tue, 30 Jun 2026 17:44:16 GMT
- Title: Human-as-Humanoid: Enabling Zero-Shot Humanoid Learning from Ego-Exo Human Videos with Human-Aligned Embodiments
- Abstract summary: Human-as-Humanoid is a human-to-humanoid supervision framework for high-DoF humanoids.<n>It converts large-scale human demonstrations into executable observation--action supervision for the target humanoid.<n>Human-as-Humanoid yields a 4.8--7.2x raw demonstration-language gain over humanoid teleoperation in our data-collection analysis.
- Score: 32.61025334275917
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Vision-language-action (VLA) models across robot embodiments require high-quality observation--action supervision to learn deployable action distributions, yet scaling such robot data remains difficult, especially for high-DoF humanoids. Teleoperation provides controller-aligned supervision, while human egocentric videos capture diverse bimanual manipulation but do not directly provide executable robot actions. We introduce Human-as-Humanoid, a human-to-humanoid supervision framework that enables near-real-time human-centric action generation, making human demonstrations usable for high-DoF humanoid VLA training by jointly aligning the robot embodiment, the sensing setup, and the action-label interface. Built on PrimeU, a human-aligned 60-DoF upper-body humanoid, Human-as-Humanoid uses synchronized ego-exo videos to pair deployment-aligned egocentric observations with exocentric motion recovery, retargets the recovered human motion through staged Inverse Kinematics (IK) into controller-aligned 60-DoF action chunks, and trains the VLA model with Forward Kinematics (FK)-aware supervision to preserve wrist and fingertip task-space geometry. This converts large-scale human demonstrations from visual observations into executable observation--action supervision for the target humanoid. Experiments validate the conversion chain at the motion-recovery, robot-action-space, and real-robot deployment levels. Human-as-Humanoid yields a 4.8--7.2x raw demonstration-throughput gain over humanoid teleoperation in our data-collection analysis, and on several downstream tasks, policies post-trained only with the converted human labels generalize to real-robot deployment without target-task robot demonstrations. The official project website is available at https://zgc-embodyai.github.io/Human-as-Humanoid.
Related papers
- HumanoidUMI: Bridging Robot-Free Demonstrations and Humanoid Whole-Body Manipulation [6.733928872257592]
We propose HumanoidUMI, a portable and robot-free framework for humanoid whole-body data collection.<n>We use lightweight VR devices and UMI-inspired grippers to collect sparse human keypoint trajectories, wrist-view observations, and gripper actions.<n>Experiments in five real-world scenarios demonstrate the effectiveness of the proposed framework.
arXiv Detail & Related papers (2026-06-25T16:23:56Z) - ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretraining [40.678451625177026]
Vision-Language-Action (VLA) models benefit from large-scale and diverse embodied data, yet scaling robot trajectory collection is costly and labor-intensive.<n>Recent advances show that large-scale egocentric human videos provide complementary real-world supervision in pretraining.<n>We introduce ACE-EGO-0, a unified VLA pretraining framework jointly leveraging heterogeneous data sources.
arXiv Detail & Related papers (2026-06-15T18:40:18Z) - HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos [58.9564236347451]
HumanEgo is a framework that bridges the embodiment gap between human and robot.<n>It lifts each human demonstration to an entity-level representation of hand-object interaction.<n>HumanEgo is robot-data-free, hardware-agnostic, data-efficient, and zero-shot human-to-robot transferable.
arXiv Detail & Related papers (2026-05-24T08:26:41Z) - ZeroWBC: Learning Natural Visuomotor Humanoid Control Directly from Human Egocentric Video [52.78703020909145]
We introduce ZeroWBC, a novel framework that learns a natural humanoid visuomotor control policy directly from human egocentric videos.<n>Our approach first fine-tunes a Vision-Language Model (VLM) to predict future whole-body human motions based on text instructions and egocentric visual context.<n>Experiments on the Unitree G1 humanoid robot demonstrate that our method outperforms baseline approaches in motion naturalness and versatility.
arXiv Detail & Related papers (2026-03-10T04:19:43Z) - EgoHumanoid: Unlocking In-the-Wild Loco-Manipulation with Robot-Free Egocentric Demonstration [67.13034606664333]
EgoHumanoid is the first framework to co-train a vision-language-action policy using egocentric human demonstrations.<n>A portable system for scalable human data collection is developed.
arXiv Detail & Related papers (2026-02-10T18:59:03Z) - FRoM-W1: Towards General Humanoid Whole-Body Control with Language Instructions [147.04372611893032]
We present FRoM-W1, an open-source framework designed to achieve general humanoid whole-body motion control using natural language.<n>We extensively evaluate FRoM-W1 on Unitree H1 and G1 robots.<n>Results demonstrate superior performance on the HumanML3D-X benchmark for human whole-body motion generation.
arXiv Detail & Related papers (2026-01-19T07:59:32Z) - MiVLA: Towards Generalizable Vision-Language-Action Model with Human-Robot Mutual Imitation Pre-training [102.850162490626]
We propose MiVLA, a vision-language-action model empowered by human-robot mutual imitation pre-training.<n>We show that MiVLA achieves strong improved generalization capability, outperforming state-of-the-art VLAs.
arXiv Detail & Related papers (2025-12-17T12:59:41Z) - EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos [49.820119587446655]
In this paper, we explore training Vision-Language-Action (VLA) models using egocentric human videos.<n>With a VLA trained on human video that predicts human wrist and hand actions, we can perform Inverse Kinematics and convert the human actions to robot actions.<n>We propose a simulation benchmark called Ego Humanoid Manipulation Benchmark, where we design diverse bimanual manipulation tasks with demonstrations.
arXiv Detail & Related papers (2025-07-16T17:27:44Z) - Learning from Massive Human Videos for Universal Humanoid Pose Control [46.417054298537195]
This paper introduces Humanoid-X, a large-scale dataset of over 20 million humanoid robot poses with corresponding text-based motion descriptions.<n>We train a large humanoid model, UH-1, which takes text instructions as input and outputs corresponding actions to control a humanoid robot.<n>Our scalable training approach leads to superior generalization in text-based humanoid control, marking a significant step toward adaptable, real-world-ready humanoid robots.
arXiv Detail & Related papers (2024-12-18T18:59:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.