DemoBridge: A Simulation-in-the-Loop Toolkit for Single-View Human Demonstration Retargeting

• URL: http://arxiv.org/abs/2607.09519v1
• Date: Fri, 10 Jul 2026 15:31:04 GMT
• Title: DemoBridge: A Simulation-in-the-Loop Toolkit for Single-View Human Demonstration Retargeting
• Abstract summary: We present DemoBridge, a toolkit that turns a single-view RGB stereo recording of a human hand demonstration into an executable, physics-validated robot-arm trajectory.<n>At the core of DemoBridge is a single collision-aware planner.<n>A physics simulator runs in the loop. It validates each phase as it is produced and backtracks on failure, so a demonstration that cannot be reproduced as given is re-planned rather than discarded.
• Score: 49.70827726228219
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present DemoBridge, an toolkit that turns a single-view RGB stereo recording of a human hand demonstration into an executable, physics-validated robot-arm trajectory. Retargeting across the embodiment gap is hard. A robot arm reaches a target with a long, articulated body whose links carry far more collision volume than a hand. Solving inverse kinematics for the mapped end-effector pose often yields no collision-free solution, and a trajectory imposes this at every waypoint. A single view adds noise, leaving the demonstrated reference inaccurate. At the core of DemoBridge is a single collision-aware planner. It optimizes the whole joint trajectory at once, reasoning jointly over alternative grasp poses, whole-arm and grasped-object collision, and fidelity to the demonstrated path. A physics simulator runs in the loop. It validates each phase as it is produced and backtracks on failure, so a demonstration that cannot be reproduced as given is re-planned rather than discarded. The resulting action sequence is dynamically stable and faithful to the demonstrated manipulation. It also doubles as a ready-to-use simulation rollout for policy learning. Grasp timing is inferred automatically, and the perception backends, robot, and pipeline stages are swappable from configuration. We evaluate whole-pipeline retargeting on three real-demonstration tasks and the planner on a controlled synthetic benchmark. Our code is available at https://gitlab.kuleuven.be/u0123974/demo-bridge/ .

Related papers

• Collision-Aware Humanoid Whole-Body Control under Imperfect Tracking Targets [11.010074477825272]
  Whole-body controllers (WBCs) execute commands that track targets while maintaining balance and stability.<n>Most WBCs are blind to scene geometry, which can lead to collisions due to perception, planning, or teleoperation errors.<n>We propose RECAL, a layer that wraps a blind WBC to trade off target tracking against collision avoidance using external scene geometry.
  arXiv  Detail & Related papers  (2026-09-14T22:22:02Z)
• KineBench: Benchmarking Embodied World Models via IDM-Free Kinematic Grounding [58.541098216700796]
  evaluating the physical consistency of embodied world models is a critical open challenge.<n>Existing frameworks almost exclusively rely on Inverse Dynamics Models(IDMs) for action extraction.<n>We present KineBench, an IDM-free closed-loop benchmark for EWMs.
  arXiv  Detail & Related papers  (2026-07-22T08:04:17Z)
• WARP: Whole-Body Retargeting for Learning from Offline Human Demonstrations [6.099593815020076]
  Whole-body-Aware Retargeting from human Pose (WARP)<n>We present Whole-body-Aware Retargeting from human Pose (WARP)
  arXiv  Detail & Related papers  (2026-06-29T08:17:31Z)
• Supervise What Survives: Geometry-Guided VLA Adaptation from Synthetic Robot Videos [43.32573764638152]
  We argue that deriving low-level control from generated visuals is a mismatched abstraction.<n>We propose textbfGRA, which extracts the geometric content as future 2D end-effector waypoints.<n>On real-robot tasks, GRA outperforms pseudo-action baselines under matched data budgets.
  arXiv  Detail & Related papers  (2026-06-23T11:35:13Z)
• M3imic: Learning a Versatile Whole-Body Controller for Multimodal Motion Mimicking [45.636871702422134]
  Multi-Modal Mimic is a versatile whole-body control framework for humanoid robots.<n>It unifies heterogeneous motion reference modalities, including robot joint angles, human pose trajectories, and end-effector poses.<n>It achieves sim-to-real transfer across multiple motion reference modalities without modality-specific retraining.
  arXiv  Detail & Related papers  (2026-06-03T12:52:37Z)
• Imagine2Real: Towards Zero-shot Humanoid-Object Interaction via Video Generative Priors [51.096845970243855]
  Whole-body Humanoid-Object Interaction (HOI) is bottlenecked by the scarcity of high-fidelity 3D data.<n>We propose Imagine2Real, a zero-shot HOI framework for flexible, geometry-free interaction.
  arXiv  Detail & Related papers  (2026-05-21T10:15:39Z)
• Ada3Drift: Adaptive Training-Time Drifting for One-Step 3D Visuomotor Robotic Manipulation [53.750389076941396]
  Diffusion-based visuomotor policies capture multimodal action distributions through iterative denoising, but their high inference latency limits real-time robotic control.<n>We propose Ada3Drift, which learns a training-time drifting field that attracts predicted actions toward expert demonstration modes.<n>Ada3Drift achieves state-of-the-art performance while requiring $10times$ fewer function evaluations than diffusion-based alternatives.
  arXiv  Detail & Related papers  (2026-03-12T14:36:53Z)
• FeasibleCap: Real-Time Embodiment Constraint Guidance for In-the-Wild Robot Demonstration Collection [3.8103821995386356]
  FeasibleCap is a gripper-in-hand data collection system that brings real-time executability guidance into robot-free capture.<n>On pick-and-place and tossing tasks, FeasibleCap improves replay success and reduces the fraction of infeasible frames.
  arXiv  Detail & Related papers  (2026-03-08T10:38:43Z)
• PoseDiff: A Unified Diffusion Model Bridging Robot Pose Estimation and Video-to-Action Control [67.17998939712326]
  We present PoseDiff, a conditional diffusion model that unifies robot state estimation and control within a single framework.<n>At its core, PoseDiff maps raw visual observations into structured robot states-such as 3D keypoints or joint angles-from a single RGB image.<n>Building upon this foundation, PoseDiff extends naturally to video-to-action inverse dynamics.
  arXiv  Detail & Related papers  (2025-09-29T10:55:48Z)
• Robot See Robot Do: Imitating Articulated Object Manipulation with Monocular 4D Reconstruction [51.49400490437258]
  This work develops a method for imitating articulated object manipulation from a single monocular RGB human demonstration. We first propose 4D Differentiable Part Models (4D-DPM), a method for recovering 3D part motion from a monocular video. Given this 4D reconstruction, the robot replicates object trajectories by planning bimanual arm motions that induce the demonstrated object part motion. We evaluate 4D-DPM's 3D tracking accuracy on ground truth annotated 3D part trajectories and RSRD's physical execution performance on 9 objects across 10 trials each on a bimanual YuMi robot.
  arXiv  Detail & Related papers  (2024-09-26T17:57:16Z)
• Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulation [65.46610405509338]
  We seek to learn a generalizable goal-conditioned policy that enables zero-shot robot manipulation. Our framework,Track2Act predicts tracks of how points in an image should move in future time-steps based on a goal. We show that this approach of combining scalably learned track prediction with a residual policy enables diverse generalizable robot manipulation.
  arXiv  Detail & Related papers  (2024-05-02T17:56:55Z)
• DITTO: Demonstration Imitation by Trajectory Transformation [31.930923345163087]
  In this work, we address the problem of one-shot imitation from a single human demonstration, given by an RGB-D video recording. We propose a two-stage process. In the first stage we extract the demonstration trajectory offline. This entails segmenting manipulated objects and determining their relative motion in relation to secondary objects such as containers. In the online trajectory generation stage, we first re-detect all objects, then warp the demonstration trajectory to the current scene and execute it on the robot.
  arXiv  Detail & Related papers  (2024-03-22T13:46:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.