Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretraining

• URL: http://arxiv.org/abs/2506.16475v2
• Date: Mon, 07 Jul 2025 17:59:32 GMT
• Title: Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretraining
• Abstract summary: We introduce a cross-embodiment imitation learning system for quadrupedal manipulation, leveraging data collected from both humans and LocoMan.<n>We validate our system on six real-world manipulation tasks, where it achieves an average success rate improvement of 41.9% overall.
• Score: 38.559900728422875
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Quadrupedal robots have demonstrated impressive locomotion capabilities in complex environments, but equipping them with autonomous versatile manipulation skills in a scalable way remains a significant challenge. In this work, we introduce a cross-embodiment imitation learning system for quadrupedal manipulation, leveraging data collected from both humans and LocoMan, a quadruped equipped with multiple manipulation modes. Specifically, we develop a teleoperation and data collection pipeline, which unifies and modularizes the observation and action spaces of the human and the robot. To effectively leverage the collected data, we propose an efficient modularized architecture that supports co-training and pretraining on structured modality-aligned data across different embodiments. Additionally, we construct the first manipulation dataset for the LocoMan robot, covering various household tasks in both unimanual and bimanual modes, supplemented by a corresponding human dataset. We validate our system on six real-world manipulation tasks, where it achieves an average success rate improvement of 41.9% overall and 79.7% under out-of-distribution (OOD) settings compared to the baseline. Pretraining with human data contributes a 38.6% success rate improvement overall and 82.7% under OOD settings, enabling consistently better performance with only half the amount of robot data. Our code, hardware, and data are open-sourced at: https://human2bots.github.io.

Related papers

• Pre-training Visual Dexterity in Simulation [67.17477115017896]
  We introduce a pre-training framework for dexterous manipulation that uses data entirely collected in simulation.<n>With the help of five operators, we collect 75 hours of multi-task dexterous manipulation over one week, and use it to pre-train a causal transformer on a sequence modeling objective.<n>We find that our approach outperforms training behavior cloning policies from scratch, showing that simulation teleoperation is a viable pre-training source for real-world dexterous manipulation.
  arXiv  Detail & Related papers  (2026-08-16T20:21:33Z)
• RoboTacDex: A Dexterous Visual-Tactile-Action Dataset for Humanoid Manipulation [29.49648908136145]
  RoboTacDex is a large, multi-modal, and diverse dataset of dexterous manipulation behaviors performed with a humanoid robot.<n>Built on the publicly accessible humanoid robot Unitree G1, RoboTacDex consists of 6k trajectories covering 19 tasks, 23 skills, and interactions with 22 objects.
  arXiv  Detail & Related papers  (2026-06-30T15:39:29Z)
• Duet: Dual-Robot Understanding via Efficient Teaching [26.80188836825586]
  DUET is a dual-robot learning framework for mobile manipulation.<n>For efficient data collection, we create a unified dual-embodiment synchronized VR-based teleoperation system.<n>We also develop a tracking pipeline that records human-human coordination and collaborative mobile manipulation priors.
  arXiv  Detail & Related papers  (2026-06-18T23:47:27Z)
• HumanoidMimicGen: Data Generation for Loco-Manipulation via Whole-Body Planning [52.022681285103126]
  We present HumanoidMimicGen, a method for generating humanoid legged loco-manipulation data.<n>Our method adapts contact-rich whole-body skills from a handful of source demonstrations to new states.<n>We show that whole-body visuomotor policies co-trained with data generated by HumanoidMimicGen outperform those trained only on real-world data by 20%.
  arXiv  Detail & Related papers  (2026-05-26T21:57:11Z)
• EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the World [63.2070298441355]
  EgoVerse is a collaborative platform for human data-driven robot learning.<n>We release 1,362 hours (80k episodes) of human demonstrations spanning 1,965 tasks, 240 scenes, and 2,087 unique demonstrators.<n>We conduct a large-scale study of human-to-robot transfer with experiments replicated across multiple labs, tasks, and robot embodiments.
  arXiv  Detail & Related papers  (2026-04-08T21:27:06Z)
• $Ψ_0$: An Open Foundation Model Towards Universal Humanoid Loco-Manipulation [39.811210435945924]
  We introduce an open foundation model to address challenging humanoid loco-manipulation tasks.<n>Our research identifies a critical yet often overlooked data recipe.<n>We demonstrate that pre-training on high-quality egocentric human manipulation data followed by post-training on domain-specific real-world humanoid trajectories yields superior performance.
  arXiv  Detail & Related papers  (2026-03-12T17:59:51Z)
• CEDex: Cross-Embodiment Dexterous Grasp Generation at Scale from Human-like Contact Representations [53.37721117405022]
  Cross-embodiment dexterous grasp synthesis refers to adaptively generating and optimizing grasps for various robotic hands.<n>We propose CEDex, a novel cross-embodiment dexterous grasp synthesis method at scale.<n>We construct the largest cross-embodiment grasp dataset to date, comprising 500K objects across four types with 20M total grasps.
  arXiv  Detail & Related papers  (2025-09-29T12:08:04Z)
• H-RDT: Human Manipulation Enhanced Bimanual Robotic Manipulation [27.585828712261232]
  H-RDT (Human to Robotics Diffusion Transformer) is a novel approach that leverages human manipulation data to enhance robot manipulation capabilities.<n>Our key insight is that large-scale egocentric human manipulation videos with paired 3D hand pose annotations provide rich behavioral priors that capture natural manipulation strategies.<n>We introduce a two-stage training paradigm: (1) pre-training on large-scale egocentric human manipulation data, and (2) cross-embodiment fine-tuning on robot-specific data with modular action encoders and decoders.
  arXiv  Detail & Related papers  (2025-07-31T13:06:59Z)
• RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation [51.86515213749527]
  We present RoboTwin 2.0, a scalable simulation framework that enables automated, large-scale generation of diverse and realistic data.<n>To improve sim-to-real transfer, RoboTwin 2.0 incorporates structured domain randomization along five axes: clutter, lighting, background, tabletop height and language instructions.<n>We instantiate this framework across 50 dual-arm tasks spanning five robot embodiments, and pre-collect over 100,000 domain-randomized expert trajectories.
  arXiv  Detail & Related papers  (2025-06-22T16:26:53Z)
• Tool-as-Interface: Learning Robot Policies from Human Tool Usage through Imitation Learning [16.394434999046293]
  We propose a framework to transfer tool-use knowledge from humans to robots.<n>We validate our approach on diverse real-world tasks, including meatball scooping, pan flipping, wine bottle balancing, and other complex tasks.
  arXiv  Detail & Related papers  (2025-04-06T20:40:19Z)
• ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning [24.675197489823898]
  We introduce ManipTrans, a novel method for transferring human bimanual skills to dexterous robotic hands in simulation.<n>Experiments show that ManipTrans surpasses state-of-the-art methods in success rate, fidelity, and efficiency.<n>We also create DexManipNet, a large-scale dataset featuring previously unexplored tasks like pen capping and bottle unscrewing.
  arXiv  Detail & Related papers  (2025-03-27T17:50:30Z)
• DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning [42.88605563822155]
  We present a large-scale automated data generation system that synthesizes trajectories from human demonstrations for humanoid robots with dexterous hands.<n>We generate 21K demos across these tasks from just 60 source human demos.<n>We also present a real-to-sim-to-real pipeline and deploy it on a real-world humanoid can sorting task.
  arXiv  Detail & Related papers  (2024-10-31T17:48:45Z)
• Robots Pre-train Robots: Manipulation-Centric Robotic Representation from Large-Scale Robot Datasets [24.77850617214567]
  We propose a foundation representation learning framework capturing both visual features and the dynamics information such as actions and proprioceptions of manipulation tasks. Specifically, we pre-train a visual encoder on the DROID robotic dataset and leverage motion-relevant data such as robot proprioceptive states and actions. We introduce a novel contrastive loss that aligns visual observations with the robot's proprioceptive state-action dynamics, combined with a behavior cloning (BC)-like actor loss to predict actions during pre-training, along with a time contrastive loss.
  arXiv  Detail & Related papers  (2024-10-29T17:58:13Z)
• Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisition [48.65867987106428]
  We introduce a novel system for joint learning between human operators and robots. It enables human operators to share control of a robot end-effector with a learned assistive agent. It reduces the need for human adaptation while ensuring the collected data is of sufficient quality for downstream tasks.
  arXiv  Detail & Related papers  (2024-06-29T03:37:29Z)
• BridgeData V2: A Dataset for Robot Learning at Scale [73.86688388408021]
  BridgeData V2 is a large and diverse dataset of robotic manipulation behaviors. It contains 60,096 trajectories collected across 24 environments on a publicly available low-cost robot.
  arXiv  Detail & Related papers  (2023-08-24T17:41:20Z)
• Surfer: Progressive Reasoning with World Models for Robotic Manipulation [51.26109827779267]
  We introduce a novel and simple robot manipulation framework, called Surfer. Surfer treats robot manipulation as a state transfer of the visual scene, and decouples it into two parts: action and scene. It is based on the world model, treats robot manipulation as a state transfer of the visual scene, and decouples it into two parts: action and scene.
  arXiv  Detail & Related papers  (2023-06-20T07:06:04Z)
• Learning Predictive Models From Observation and Interaction [137.77887825854768]
  Learning predictive models from interaction with the world allows an agent, such as a robot, to learn about how the world works. However, learning a model that captures the dynamics of complex skills represents a major challenge. We propose a method to augment the training set with observational data of other agents, such as humans.
  arXiv  Detail & Related papers  (2019-12-30T01:10:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.