Functional Force-Aware Retargeting from Virtual Human Demos to Soft Robot Policies

• URL: http://arxiv.org/abs/2604.01224v1
• Date: Wed, 01 Apr 2026 17:59:35 GMT
• Title: Functional Force-Aware Retargeting from Virtual Human Demos to Soft Robot Policies
• Abstract summary: We introduce SoftAct, a framework for teaching soft robot hands to perform human-like manipulation skills by explicitly reasoning about contact forces.<n>Unlike conventional approaches that retarget human joint trajectories, SoftAct employs two-stage, force-aware algorithm.
• Score: 22.000828900701237
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce SoftAct, a framework for teaching soft robot hands to perform human-like manipulation skills by explicitly reasoning about contact forces. Leveraging immersive virtual reality, our system captures rich human demonstrations, including hand kinematics, object motion, dense contact patches, and detailed contact force information. Unlike conventional approaches that retarget human joint trajectories, SoftAct employs a two-stage, force-aware retargeting algorithm. The first stage attributes demonstrated contact forces to individual human fingers and allocates robot fingers proportionally, establishing a force-balanced mapping between human and robot hands. The second stage performs online retargeting by combining baseline end-effector pose tracking with geodesic-weighted contact refinements, using contact geometry and force magnitude to adjust robot fingertip targets in real time. This formulation enables soft robotic hands to reproduce the functional intent of human demonstrations while naturally accommodating extreme embodiment mismatch and nonlinear compliance. We evaluate SoftAct on a suite of contact-rich manipulation tasks using a custom non-anthropomorphic pneumatic soft robot hand. SoftAct's controller reduces fingertip trajectory tracking RMSE by up to 55 percent and reduces tracking variance by up to 69 percent compared to kinematic and learning-based baselines. At the policy level, SoftAct achieves consistently higher success in zero-shot real-world deployment and in simulation. These results demonstrate that explicitly modeling contact geometry and force distribution is essential for effective skill transfer to soft robotic hands, and cannot be recovered through kinematic imitation alone. Project videos and additional details are available at https://soft-act.github.io/.

Related papers

• Unified Motion Retargeting for Humanoids with Learned Point Cloud Correspondence [24.060385614379783]
  We present Unified Motion Retargeting (UMR), a framework that learns dense point cloud correspondence without requiring manually designed human-robot mappings.<n>UMR achieves higher motion fidelity and plausibility than state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-09-02T05:43:19Z)
• RGB-D Video Generation for Improving Human-to-Robot Object Handover Prediction [51.76737991200891]
  Human-to-robot (H2R) object handover is a fundamental capability for human-robot collaboration, yet progress is hindered by the scarcity of large-scale, human-centric datasets.<n>We introduce Hand2Bot, an RGB-D video dataset that provides rich contextual information such as body posture and facial expressions.<n>We propose PassGen, a generative pipeline that synthesizes realistic handover sequences while ensuring hand-object consistency.
  arXiv  Detail & Related papers  (2026-08-13T09:55:59Z)
• ObjRetarget: An Object-Aware Motion Retargeting Framework with Anthropomorphic Arm Constraints and Polyhedral Hand Modeling [9.378632224914458]
  dexterous manipulation from human manipulation videos requires reliably human intent to executable robot actions.<n>Existing methods often ignore explicit contact modeling or reinforcement, resulting in limited accuracy and generalization.<n>We propose Retarget, a human-to-robot motion framework for learning robot dexterous manipulation from human videos.
  arXiv  Detail & Related papers  (2026-07-04T11:31:23Z)
• Do Rigid-Body Simulators Dream of Soft Robots? Learning Contact-Rich Manipulation for Tendon-Driven Continuum Robots [4.338766069286991]
  We bridge the gap for tendon-driven continuum robots by deriving a continuum-mechanics-informed discretization.<n>This is the first demonstration of sim-to-real transfer for contact-rich manipulation with continuum robots.
  arXiv  Detail & Related papers  (2026-06-21T09:02:53Z)
• FRoM-W1: Towards General Humanoid Whole-Body Control with Language Instructions [147.04372611893032]
  We present FRoM-W1, an open-source framework designed to achieve general humanoid whole-body motion control using natural language.<n>We extensively evaluate FRoM-W1 on Unitree H1 and G1 robots.<n>Results demonstrate superior performance on the HumanML3D-X benchmark for human whole-body motion generation.
  arXiv  Detail & Related papers  (2026-01-19T07:59:32Z)
• Semantic Co-Speech Gesture Synthesis and Real-Time Control for Humanoid Robots [5.531678625546847]
  We present an innovative end-to-end framework for synthesizing semantically meaningful co-speech gestures and deploying them in real-time on a humanoid robot.<n>Our core innovation lies in the meticulous integration of a semantics-aware gesture synthesis module.<n>We demonstrate that our combined system produces semantically appropriate and rhythmically coherent gestures.
  arXiv  Detail & Related papers  (2025-12-19T02:55:10Z)
• OSMO: Open-Source Tactile Glove for Human-to-Robot Skill Transfer [34.13467792368733]
  We introduce OSMO, a wearable tactile glove designed for human-to-robot skill transfer.<n>We demonstrate that a robot policy trained exclusively on human demonstrations is capable of executing a challenging contact-rich manipulation task.
  arXiv  Detail & Related papers  (2025-12-09T18:56:30Z)
• ResMimic: From General Motion Tracking to Humanoid Whole-body Loco-Manipulation via Residual Learning [59.64325421657381]
  Humanoid whole-body loco-manipulation promises transformative capabilities for daily service and warehouse tasks.<n>We introduce ResMimic, a two-stage residual learning framework for precise and expressive humanoid control from human motion data.<n>Results show substantial gains in task success, training efficiency, and robustness over strong baselines.
  arXiv  Detail & Related papers  (2025-10-06T17:47:02Z)
• From Human Hands to Robot Arms: Manipulation Skills Transfer via Trajectory Alignment [36.08997778717271]
  Learning diverse manipulation skills for real-world robots is bottlenecked by reliance on costly and hard-to-scale teleoperated demonstrations.<n>We introduce Traj2Action, a novel framework that bridges this embodiment gap by using the 3D trajectory of the operational endpoint as a unified intermediate representation.<n>Our policy first learns to generate a coarse trajectory, which forms a high-level motion plan by leveraging both human and robot data.
  arXiv  Detail & Related papers  (2025-10-01T04:21:12Z)
• KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skills [58.73043119128804]
  This paper presents a physics-based humanoid control framework, aiming to master highly-dynamic human behaviors such as Kungfu and dancing.<n>For motion processing, we design a pipeline to extract, filter out, correct, and retarget motions, while ensuring compliance with physical constraints.<n>For motion imitation, we formulate a bi-level optimization problem to dynamically adjust the tracking accuracy tolerance.<n>In experiments, we train whole-body control policies to imitate a set of highly-dynamic motions.
  arXiv  Detail & Related papers  (2025-06-15T13:58:53Z)
• Feel the Force: Contact-Driven Learning from Humans [52.36160086934298]
  Controlling fine-grained forces during manipulation remains a core challenge in robotics.<n>We present FeelTheForce, a robot learning system that models human tactile behavior to learn force-sensitive manipulation.<n>Our approach grounds robust low-level force control in scalable human supervision, achieving a 77% success rate across 5 force-sensitive manipulation tasks.
  arXiv  Detail & Related papers  (2025-06-02T17:57:52Z)
• Hierarchical Procedural Framework for Low-latency Robot-Assisted Hand-Object Interaction [45.256762954338704]
  We propose a hierarchical procedural framework to enable robot-assisted hand-object interaction (HOI)<n>An open-loop hierarchy leverages the RGB-based 3D reconstruction of the human hand, based on which motion primitives have been designed to translate hand motions into robotic actions.<n>A case study of ring-wearing tasks indicates the potential application of this work in assistive technologies such as healthcare and manufacturing.
  arXiv  Detail & Related papers  (2024-05-29T21:20:16Z)
• HERD: Continuous Human-to-Robot Evolution for Learning from Human Demonstration [57.045140028275036]
  We show that manipulation skills can be transferred from a human to a robot through the use of micro-evolutionary reinforcement learning. We propose an algorithm for multi-dimensional evolution path searching that allows joint optimization of both the robot evolution path and the policy.
  arXiv  Detail & Related papers  (2022-12-08T15:56:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.