UniT: Toward a Unified Physical Language for Human-to-Humanoid Policy Learning and World Modeling

• URL: http://arxiv.org/abs/2604.19734v1
• Date: Tue, 21 Apr 2026 17:57:27 GMT
• Title: UniT: Toward a Unified Physical Language for Human-to-Humanoid Policy Learning and World Modeling
• Abstract summary: We introduce UniT, a framework that establishes a unified physical language for human-to-humanoid transfer.<n>UniT employs a tri-branch cross-reconstruction mechanism: actions predict vision to anchor kinematics to physical outcomes, while vision reconstructs actions to filter out irrelevant visual confounders.
• Score: 41.44664297185315
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scaling humanoid foundation models is bottlenecked by the scarcity of robotic data. While massive egocentric human data offers a scalable alternative, bridging the cross-embodiment chasm remains a fundamental challenge due to kinematic mismatches. We introduce UniT (Unified Latent Action Tokenizer via Visual Anchoring), a framework that establishes a unified physical language for human-to-humanoid transfer. Grounded in the philosophy that heterogeneous kinematics share universal visual consequences, UniT employs a tri-branch cross-reconstruction mechanism: actions predict vision to anchor kinematics to physical outcomes, while vision reconstructs actions to filter out irrelevant visual confounders. Concurrently, a fusion branch synergies these purified modalities into a shared discrete latent space of embodiment-agnostic physical intents. We validate UniT across two paradigms: 1) Policy Learning (VLA-UniT): By predicting these unified tokens, it effectively leverages diverse human data to achieve state-of-the-art data efficiency and robust out-of-distribution (OOD) generalization on both humanoid simulation benchmark and real-world deployments, notably demonstrating zero-shot task transfer. 2) World Modeling (WM-UniT): By aligning cross-embodiment dynamics via unified tokens as conditions, it realizes direct human-to-humanoid action transfer. This alignment ensures that human data seamlessly translates into enhanced action controllability for humanoid video generation. Ultimately, by inducing a highly aligned cross-embodiment representation (empirically verified by t-SNE visualizations revealing the convergence of human and humanoid features into a shared manifold), UniT offers a scalable path to distill vast human knowledge into general-purpose humanoid capabilities.

Related papers

• Human2Humanoid: Physics-Aware Cross-Morphology Motion Retargeting for Humanoid Robots [9.096721498763717]
  Retargeting human motion to humanoid robots is critical for teleoperation, imitation learning and human-robot interaction.<n>This paper presents Human2Human, an unsupervised motion framework that transfers human motions to humanoid robot behaviors with high fidelity.<n> Experimental results show that the proposed method successfully retargets human motion to the Unitree G1 humanoid robot without paired data.
  arXiv  Detail & Related papers  (2026-06-02T10:53:06Z)
• LACE: Latent Visual Representation for Cross-Embodiment Learning [36.1587655731958]
  Cross-embodiment learning from human demonstrations is hindered by the visual gap between human and robot embodiments.<n>We propose LACE, a framework that aligns human and robot visual representations in the latent space of self-supervised learning backbones.
  arXiv  Detail & Related papers  (2026-05-16T01:50:18Z)
• HEX: Humanoid-Aligned Experts for Cross-Embodiment Whole-Body Manipulation [74.34984994596813]
  HEX is a state-centric framework for coordinated manipulation on humanoid robots.<n>It incorporates a humanoid-aligned universal state representation for scalable learning across heterogeneous embodiments.<n>It achieves state-of-the-art performance in task success rate and generalization.
  arXiv  Detail & Related papers  (2026-04-09T09:01:43Z)
• UniHM: Unified Dexterous Hand Manipulation with Vision Language Model [39.2419824041854]
  Planning physically feasible dexterous hand manipulation is a central challenge in robotic manipulation and Embodied AI.<n>We introduce UniHM, the first framework for unified dexterous hand manipulation guided by free-form language commands.
  arXiv  Detail & Related papers  (2026-02-28T16:37:11Z)
• Learning Whole-Body Human-Humanoid Interaction from Human-Human Demonstrations [63.80827184637476]
  We introduce D-STAR, a hierarchical policy that disentangles when to act from where to act.<n>We validate our framework through extensive and rigorous simulations.
  arXiv  Detail & Related papers  (2026-01-14T14:37:06Z)
• MiVLA: Towards Generalizable Vision-Language-Action Model with Human-Robot Mutual Imitation Pre-training [102.850162490626]
  We propose MiVLA, a vision-language-action model empowered by human-robot mutual imitation pre-training.<n>We show that MiVLA achieves strong improved generalization capability, outperforming state-of-the-art VLAs.
  arXiv  Detail & Related papers  (2025-12-17T12:59:41Z)
• UniBYD: A Unified Framework for Learning Robotic Manipulation Across Embodiments Beyond Imitation of Human Demonstrations [35.77665515297785]
  In embodied intelligence, the embodiment gap between robotic and human hands brings significant challenges for learning from human demonstrations.<n>We propose UniBYD, a unified framework that uses a dynamic reinforcement learning algorithm to discover manipulation policies aligned with the robot's physical characteristics.<n>To evaluate UniBYD, we propose UniManip, the first benchmark for cross-embodiment manipulation spanning diverse robotic morphologies.
  arXiv  Detail & Related papers  (2025-12-12T14:48:06Z)
• ECHO: Ego-Centric modeling of Human-Object interactions [71.17118015822699]
  ECHO (Ego-Centric modeling of Human-Object interactions) is developed.<n>It recovers three modalities: human pose, object motion, and contact from such minimal observation.<n>It outperforms existing methods that do not offer the same flexibility.
  arXiv  Detail & Related papers  (2025-08-29T12:12:22Z)
• CoopDiff: Anticipating 3D Human-object Interactions via Contact-consistent Decoupled Diffusion [62.93198247045824]
  3D human-object interaction (HOI) anticipation aims to predict the future motion of humans and their manipulated objects, conditioned on the historical context.<n>We propose a novel contact-consistent decoupled diffusion framework CoopDiff, which employs two distinct branches to decouple human and object motion modeling.
  arXiv  Detail & Related papers  (2025-08-10T03:29:17Z)
• HumanSAM: Classifying Human-centric Forgery Videos in Human Spatial, Appearance, and Motion Anomaly [15.347208661111198]
  HumanSAM aims to classify humancentric forgeries into three distinct types of artifacts commonly observed in generated content.<n>HumanSAM yields promising results in comparison with state-of-the-art methods, both in binary and multi-class forgery classification.
  arXiv  Detail & Related papers  (2025-07-26T12:03:47Z)
• TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild [77.59069361196404]
  TRiPOD is a novel method for predicting body dynamics based on graph attentional networks. To incorporate a real-world challenge, we learn an indicator representing whether an estimated body joint is visible/invisible at each frame. Our evaluation shows that TRiPOD outperforms all prior work and state-of-the-art specifically designed for each of the trajectory and pose forecasting tasks.
  arXiv  Detail & Related papers  (2021-04-08T20:01:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.