OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulation

• URL: http://arxiv.org/abs/2606.22174v1
• Date: Sat, 20 Jun 2026 18:02:50 GMT
• Title: OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulation
• Abstract summary: Whole-body humanoid loco-manipulation requires coordinating the robot's entire kinematic chain.<n>We build a whole-body native vision-variable-action model that maps language and pixels directly to all of the humanoid's degrees of freedom.<n>Following this roadmap yields OpenHLM, an open-source recipe for whole-body humanoid loco-manipulation.
• Score: 22.18805694472118
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Whole-body humanoid loco-manipulation requires coordinating the robot's entire kinematic chain. However, most existing systems typically decouple the upper and lower bodies into separate controllers, limiting such coordination and yielding behaviors similar to those of a wheeled dual-arm platform. In this paper, we ask what it takes to build a whole-body native vision-language-action (VLA) model that maps language and pixels directly to all of the humanoid's degrees of freedom. We conduct a systematic empirical study organized as a roadmap of one-variable-at-a-time experiments across three phases: whole-body teleoperation, VLA model design, and heterogeneous co-training. Our study yields several intriguing findings: a joint-based whole-body teleoperation interface outperforms alternatives that only partially expose the humanoid's degrees of freedom; a VLA pretrained on static and wheeled dual-arm platforms transfers surprisingly well to a humanoid's full action space; and co-training with HuMI, the humanoid analog of UMI, extends the policy to new objects and instructions without additional whole-body teleoperation on those targets. Following this roadmap yields OpenHLM, an open-source recipe for whole-body humanoid loco-manipulation. In a challenging long-horizon task that spans a wide vertical range of the humanoid, OpenHLM outperforms two state-of-the-art humanoid VLA baselines (GR00T N1.6 and $Ψ_0$) using less than half the total demonstration time. Our code, training data, and model checkpoints are available at [https://openhlm-project.github.io/].

Related papers

• Human-as-Humanoid: Enabling Zero-Shot Humanoid Learning from Ego-Exo Human Videos with Human-Aligned Embodiments [32.61025334275917]
  Human-as-Humanoid is a human-to-humanoid supervision framework for high-DoF humanoids.<n>It converts large-scale human demonstrations into executable observation--action supervision for the target humanoid.<n>Human-as-Humanoid yields a 4.8--7.2x raw demonstration-language gain over humanoid teleoperation in our data-collection analysis.
  arXiv  Detail & Related papers  (2026-06-30T17:44:16Z)
• VLK: Learning Humanoid Loco-Manipulation from Synthetic Interactions in Reconstructed Scenes [71.14444761145435]
  Perception-based humanoid loco-manipulation requires connecting egocentric observations and task instructions to whole-body motion.<n>No existing data source provides this complete at scale.<n>We address this bottleneck by generating vision-language-kinematics supervision synthetically in reconstructed scenes.
  arXiv  Detail & Related papers  (2026-06-29T17:59:55Z)
• HumanoidUMI: Bridging Robot-Free Demonstrations and Humanoid Whole-Body Manipulation [6.733928872257592]
  We propose HumanoidUMI, a portable and robot-free framework for humanoid whole-body data collection.<n>We use lightweight VR devices and UMI-inspired grippers to collect sparse human keypoint trajectories, wrist-view observations, and gripper actions.<n>Experiments in five real-world scenarios demonstrate the effectiveness of the proposed framework.
  arXiv  Detail & Related papers  (2026-06-25T16:23:56Z)
• ZeroWBC: Learning Natural Visuomotor Humanoid Control Directly from Human Egocentric Video [52.78703020909145]
  We introduce ZeroWBC, a novel framework that learns a natural humanoid visuomotor control policy directly from human egocentric videos.<n>Our approach first fine-tunes a Vision-Language Model (VLM) to predict future whole-body human motions based on text instructions and egocentric visual context.<n>Experiments on the Unitree G1 humanoid robot demonstrate that our method outperforms baseline approaches in motion naturalness and versatility.
  arXiv  Detail & Related papers  (2026-03-10T04:19:43Z)
• ULTRA: Unified Multimodal Control for Autonomous Humanoid Whole-Body Loco-Manipulation [55.467742403416175]
  We introduce a physics-driven neural algorithm that translates large-scale motion capture to humanoid embodiments.<n>We learn a unified multimodal controller that supports both dense references and sparse task specifications.<n>Results show that ULTRA generalizes to autonomous, goal-conditioned whole-body loco-manipulation from egocentric perception.
  arXiv  Detail & Related papers  (2026-03-03T18:59:29Z)
• HoMMI: Learning Whole-Body Mobile Manipulation from Human Demonstrations [15.551927664158695]
  Whole-Body Mobile Manipulation Interface (HoMMI) learns whole-body mobile manipulation directly from robot-free human demonstrations.<n>We augment UMI interfaces with egocentric sensing to capture the global context required for mobile manipulation.<n>We explicitly bridge this gap with a cross-embodiment hand-eye policy design.
  arXiv  Detail & Related papers  (2026-03-03T18:36:49Z)
• Emergence of Human to Robot Transfer in Vision-Language-Action Models [88.76648919814771]
  Vision-language-action (VLA) models can enable broad open world generalization, but require large and diverse datasets.<n>We show that human-to-robot transfer emerges once the VLA is pre-trained on sufficient scenes, tasks, and embodiments.
  arXiv  Detail & Related papers  (2025-12-27T00:13:11Z)
• TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection System [79.48781507497769]
  Existing humanoid teleoperation systems either use decoupled control or depend on expensive motion capture setups.<n>We introduce TWIST2, a portable, mocap-free humanoid teleoperation and data collection system.<n>We demonstrate long-horizon dexterous and mobile humanoid skills and we can collect 100 demonstrations in 15 minutes with an almost 100% success rate.
  arXiv  Detail & Related papers  (2025-11-04T18:58:35Z)
• DemoHLM: From One Demonstration to Generalizable Humanoid Loco-Manipulation [29.519071338337685]
  We present DemoHLM, a framework for humanoid loco-manipulation on a real humanoid robot from a single demonstration in simulation.<n>whole-body controller maps whole-body motion commands to joint torques and provides omnidirectional mobility for the humanoid robot.<n> Experiments show a positive correlation between the amount of synthetic data and policy performance.
  arXiv  Detail & Related papers  (2025-10-13T10:49:40Z)
• HumanoidVerse: A Versatile Humanoid for Vision-Language Guided Multi-Object Rearrangement [51.16740261131198]
  We introduce HumanoidVerse, a novel framework for vision-language guided humanoid control.<n>HumanoidVerse supports consecutive manipulation of multiple objects, guided only by natural language instructions and egocentric camera RGB observations.<n>Our work represents a key step toward robust, general-purpose humanoid agents capable of executing complex, sequential tasks under real-world sensory constraints.
  arXiv  Detail & Related papers  (2025-08-23T08:23:14Z)
• HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpit [52.12750762494588]
  This paper introduces HOMIE, a semi-autonomous teleoperation system.<n>It combines a reinforcement learning policy for body control mapped to a pedal, an isomorphic exoskeleton arm for arm control, and motion-sensing gloves for hand control.<n>The system is fully open-source, demos and code can be found in our https://homietele.org/.
  arXiv  Detail & Related papers  (2025-02-18T16:33:38Z)
• OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning [45.51662378032706]
  We present OmniH2O, a learning-based system for whole-body humanoid teleoperation and autonomy. Using kinematic as a universal control interface, OmniH2O enables various ways for a human to control a full-sized humanoid with dexterous hands. We release the first humanoid whole-body control dataset, OmniH2O-6, containing six everyday tasks, and demonstrate humanoid whole-body skill learning from teleoperated datasets.
  arXiv  Detail & Related papers  (2024-06-13T06:44:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.