WARP: Whole-Body Retargeting for Learning from Offline Human Demonstrations

• URL: http://arxiv.org/abs/2606.29940v1
• Date: Mon, 29 Jun 2026 08:17:31 GMT
• Title: WARP: Whole-Body Retargeting for Learning from Offline Human Demonstrations
• Abstract summary: Whole-body-Aware Retargeting from human Pose (WARP)<n>We present Whole-body-Aware Retargeting from human Pose (WARP)
• Score: 6.099593815020076
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Direct transfer from human demonstration to learnable robot action is a crucial step towards scalable whole-body mobile manipulation. While human data scales better than mobile teleoperation, it requires overcoming significant embodiment gaps. Existing retargeting methods yield imprecise or inconsistent solutions, causing action multi-modality that prevents supervised policies from reliably converging. We present Whole-body-Aware Retargeting from human Pose (WARP), an offline pipeline that explicitly models embodiment differences to extract precise, unique whole-body actions. WARP leverages a closed-form Shoulder-Elbow-Wrist (SEW) geometric solver for exact end-effector tracking while preserving whole-body structural intent. Paired with lazy mobile-base control, it extracts accurate, consistent robot trajectories. Evaluations show WARP provides highly reliable data for open-loop real-world replay. To our knowledge, WARP is the first framework to achieve zero-shot whole-body mobile manipulation directly from offline human demonstrations, eliminating the need for human-in-the-loop teleoperation action data. More details on https://warp-retarget.github.io/

Related papers

• DemoBridge: A Simulation-in-the-Loop Toolkit for Single-View Human Demonstration Retargeting [49.70827726228219]
  We present DemoBridge, a toolkit that turns a single-view RGB stereo recording of a human hand demonstration into an executable, physics-validated robot-arm trajectory.<n>At the core of DemoBridge is a single collision-aware planner.<n>A physics simulator runs in the loop. It validates each phase as it is produced and backtracks on failure, so a demonstration that cannot be reproduced as given is re-planned rather than discarded.
  arXiv  Detail & Related papers  (2026-07-10T15:31:04Z)
• HumanoidUMI: Bridging Robot-Free Demonstrations and Humanoid Whole-Body Manipulation [6.733928872257592]
  We propose HumanoidUMI, a portable and robot-free framework for humanoid whole-body data collection.<n>We use lightweight VR devices and UMI-inspired grippers to collect sparse human keypoint trajectories, wrist-view observations, and gripper actions.<n>Experiments in five real-world scenarios demonstrate the effectiveness of the proposed framework.
  arXiv  Detail & Related papers  (2026-06-25T16:23:56Z)
• Do as I Do: Dexterous Manipulation Data from Everyday Human Videos [69.98851738909168]
  We present DO AS I DO, an algorithm to reconstruct and retarget monocular RGB human videos to multi-fingered dexterous robotic hands.<n>The algorithm then retargets these hand-object interaction estimates into a sequence of actions executable in the real world.<n>Overall, DO AS I DO outperforms previous state of the art in estimating hand-object interactions and extracting dexterous manipulation trajectories from RGB videos.
  arXiv  Detail & Related papers  (2026-06-17T17:57:34Z)
• Contrastive Action-Image Pre-training for Visuomotor Control [91.5630572541332]
  We introduce CAIP, a vision encoder that treats human hand poses from large-scale egocentric video as a proxy for end-effector actions.<n>Leveraging 32,041 hours of egocentric human video and only 88 hours of robotic manipulation data, CAIP outperforms state-of-the-art vision encoders.
  arXiv  Detail & Related papers  (2026-06-15T20:00:20Z)
• LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisition [11.86733592383987]
  LUCID is a framework that learns task intent from unstructured human videos.<n>It learns robot control in massively-parallel simulation.<n>We evaluate LUCID on five real-world manipulation tasks.
  arXiv  Detail & Related papers  (2026-06-10T03:49:01Z)
• ZeroWBC: Learning Natural Visuomotor Humanoid Control Directly from Human Egocentric Video [52.78703020909145]
  We introduce ZeroWBC, a novel framework that learns a natural humanoid visuomotor control policy directly from human egocentric videos.<n>Our approach first fine-tunes a Vision-Language Model (VLM) to predict future whole-body human motions based on text instructions and egocentric visual context.<n>Experiments on the Unitree G1 humanoid robot demonstrate that our method outperforms baseline approaches in motion naturalness and versatility.
  arXiv  Detail & Related papers  (2026-03-10T04:19:43Z)
• A Closed-Form Geometric Retargeting Solver for Upper Body Humanoid Robot Teleoperation [32.82085155744172]
  Methods for targeting human motion to robot arms are suboptimal and slow.<n>This paper reframes orientation alignment as an orientation problem.<n>Key idea is to align a robot arm to a human's upper and lower arm orientations, as identified from shoulder, elbow, and wrist (SEW) keypoints.<n> Experiments show that SEW-Mimic outperforms other methods in computation time and accuracy.
  arXiv  Detail & Related papers  (2026-02-02T04:44:55Z)
• Dexplore: Scalable Neural Control for Dexterous Manipulation from Reference-Scoped Exploration [58.4036440289082]
  Hand-object motion-capture (MoCap) offer large-scale, contact-rich demonstrations and hold promise for dexterous robotic scopes.<n>We introduce Dexplore, a unified single-loop optimization that performs repositories and tracking to learn robot control policies directly from MoCap at scale.
  arXiv  Detail & Related papers  (2025-09-11T17:59:07Z)
• DemoDiffusion: One-Shot Human Imitation using pre-trained Diffusion Policy [33.18108154271181]
  We propose DemoDiffusion, a simple and scalable method for enabling robots to perform manipulation tasks in natural environments.<n>Our approach is based on two key insights. First, the hand motion in a human demonstration provides a useful prior for the robot's end-effector trajectory.<n>Second, while this retargeted motion captures the overall structure of the task, it may not align well with plausible robot actions in-context.
  arXiv  Detail & Related papers  (2025-06-25T17:59:01Z)
• Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulation [65.46610405509338]
  We seek to learn a generalizable goal-conditioned policy that enables zero-shot robot manipulation. Our framework,Track2Act predicts tracks of how points in an image should move in future time-steps based on a goal. We show that this approach of combining scalably learned track prediction with a residual policy enables diverse generalizable robot manipulation.
  arXiv  Detail & Related papers  (2024-05-02T17:56:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.