Trajectory Conditioned Cross-embodiment Skill Transfer

• URL: http://arxiv.org/abs/2510.07773v1
• Date: Thu, 09 Oct 2025 04:26:06 GMT
• Title: Trajectory Conditioned Cross-embodiment Skill Transfer
• Abstract summary: TrajSkill is a framework for Trajectory Conditioned Cross-embodiment Skill Transfer.<n>We show that TrajSkill reduces FVD by 39.6% and KVD by 36.6% compared with the state-of-the-art.
• Score: 17.361383693716643
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning manipulation skills from human demonstration videos presents a promising yet challenging problem, primarily due to the significant embodiment gap between human body and robot manipulators. Existing methods rely on paired datasets or hand-crafted rewards, which limit scalability and generalization. We propose TrajSkill, a framework for Trajectory Conditioned Cross-embodiment Skill Transfer, enabling robots to acquire manipulation skills directly from human demonstration videos. Our key insight is to represent human motions as sparse optical flow trajectories, which serve as embodiment-agnostic motion cues by removing morphological variations while preserving essential dynamics. Conditioned on these trajectories together with visual and textual inputs, TrajSkill jointly synthesizes temporally consistent robot manipulation videos and translates them into executable actions, thereby achieving cross-embodiment skill transfer. Extensive experiments are conducted, and the results on simulation data (MetaWorld) show that TrajSkill reduces FVD by 39.6\% and KVD by 36.6\% compared with the state-of-the-art, and improves cross-embodiment success rate by up to 16.7\%. Real-robot experiments in kitchen manipulation tasks further validate the effectiveness of our approach, demonstrating practical human-to-robot skill transfer across embodiments.

Related papers

• AdvDex: Learning Dexterous Manipulation from Human Demonstrations via Joint-Aligned Actions and Adversarial Learning [25.241504281004243]
  We present AdvDex, a framework for learning dexterous manipulation from human and robot demonstrations.<n>First, we introduce OmniShare, a large-scale multimodal dataset of human manipulation demonstrations.<n>Second, we propose the Joint-Aligned Action Space (JAAS), a canonical action representation comprising an $mathrmSE(3)$ wrist pose and 15 finger joints.<n>Third, we use domain-adversarial learning to reduce embodiment-specific information in the learned visual representation.
  arXiv  Detail & Related papers  (2026-08-14T07:19:06Z)
• H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Models [56.60098789696351]
  H2R-Bench is a benchmark for evaluating cross-embodiment human-to-robot manipulation video generation.<n>Each benchmark instance contains a human demonstration video, target embodiment constraints, and source-grounded annotations.<n>We benchmark eleven state-of-the-art video generation models across six manipulation families and two robot embodiments.
  arXiv  Detail & Related papers  (2026-08-13T10:14:33Z)
• Cross-Embodiment Transfer via Behavior-Aligned Representations [29.171887164748124]
  We study the role of using behavior-aligned representations in vision--action models to promote cross-embodiment transfer.<n>We develop a simulation-based benchmark designed to assess transfer with diverse cross-embodiment data to new embodiments.<n>We demonstrate that they can enhance sim-to-real cross-embodiment transfer, improving task completion progress of real robot policies pre-trained on simulation data by 28%.
  arXiv  Detail & Related papers  (2026-07-30T00:49:53Z)
• Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments [96.23886784364997]
  Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation.<n>We present Qwen-VLA, a unified embodied foundation model that extends Qwen's vision-language modeling stack to continuous action and trajectory generation.
  arXiv  Detail & Related papers  (2026-05-28T17:36:31Z)
• MiVLA: Towards Generalizable Vision-Language-Action Model with Human-Robot Mutual Imitation Pre-training [102.850162490626]
  We propose MiVLA, a vision-language-action model empowered by human-robot mutual imitation pre-training.<n>We show that MiVLA achieves strong improved generalization capability, outperforming state-of-the-art VLAs.
  arXiv  Detail & Related papers  (2025-12-17T12:59:41Z)
• StaMo: Unsupervised Learning of Generalizable Robot Motion from Compact State Representation [56.996371714721995]
  We propose an unsupervised approach that learns a highly compressed two-token state representation.<n>Our representation is efficient, interpretable, and integrates seamlessly into existing VLA-based models.<n>We name our method StaMo for its ability to learn generalizable robotic Motion from compact State representation.
  arXiv  Detail & Related papers  (2025-10-06T17:37:24Z)
• From Human Hands to Robot Arms: Manipulation Skills Transfer via Trajectory Alignment [36.08997778717271]
  Learning diverse manipulation skills for real-world robots is bottlenecked by reliance on costly and hard-to-scale teleoperated demonstrations.<n>We introduce Traj2Action, a novel framework that bridges this embodiment gap by using the 3D trajectory of the operational endpoint as a unified intermediate representation.<n>Our policy first learns to generate a coarse trajectory, which forms a high-level motion plan by leveraging both human and robot data.
  arXiv  Detail & Related papers  (2025-10-01T04:21:12Z)
• CEDex: Cross-Embodiment Dexterous Grasp Generation at Scale from Human-like Contact Representations [53.37721117405022]
  Cross-embodiment dexterous grasp synthesis refers to adaptively generating and optimizing grasps for various robotic hands.<n>We propose CEDex, a novel cross-embodiment dexterous grasp synthesis method at scale.<n>We construct the largest cross-embodiment grasp dataset to date, comprising 500K objects across four types with 20M total grasps.
  arXiv  Detail & Related papers  (2025-09-29T12:08:04Z)
• FreeAction: Training-Free Techniques for Enhanced Fidelity of Trajectory-to-Video Generation [50.39748673817223]
  We introduce two training-free, inference-time techniques that fully exploit explicit action parameters in robot video generation.<n>First, action-scaled classifier-free guidance dynamically modulates guidance strength in proportion to action magnitude, enhancing controllability over motion intensity.<n>Second, action-scaled noise truncation adjusts the distribution of initially sampled noise to better align with the desired motion dynamics.
  arXiv  Detail & Related papers  (2025-09-29T03:30:40Z)
• Joint Flow Trajectory Optimization For Feasible Robot Motion Generation from Video Demonstrations [8.133207162076877]
  We propose a framework for grasp pose generation and object trajectory imitation under the video-based Learning-from-Demonstration (LfD) paradigm.<n>Rather than directly imitating human hand motions, our method treats demonstrations as object-centric guides.<n>We validate our approach in both simulation and real-world experiments across diverse real-world manipulation tasks.
  arXiv  Detail & Related papers  (2025-09-25T03:11:07Z)
• Physical Autoregressive Model for Robotic Manipulation without Action Pretraining [65.8971623698511]
  We build upon autoregressive video generation models to propose a Physical Autoregressive Model (PAR)<n>PAR leverages the world knowledge embedded in video pretraining to understand physical dynamics without requiring action pretraining.<n>Experiments on the ManiSkill benchmark show that PAR achieves a 100% success rate on the PushCube task.
  arXiv  Detail & Related papers  (2025-08-13T13:54:51Z)
• H-RDT: Human Manipulation Enhanced Bimanual Robotic Manipulation [27.585828712261232]
  H-RDT (Human to Robotics Diffusion Transformer) is a novel approach that leverages human manipulation data to enhance robot manipulation capabilities.<n>Our key insight is that large-scale egocentric human manipulation videos with paired 3D hand pose annotations provide rich behavioral priors that capture natural manipulation strategies.<n>We introduce a two-stage training paradigm: (1) pre-training on large-scale egocentric human manipulation data, and (2) cross-embodiment fine-tuning on robot-specific data with modular action encoders and decoders.
  arXiv  Detail & Related papers  (2025-07-31T13:06:59Z)
• Is Diversity All You Need for Scalable Robotic Manipulation? [50.747150672933316]
  We investigate the nuanced role of data diversity in robot learning by examining three critical dimensions-task (what to do), embodiment (which robot to use), and expert (who demonstrates)-challenging the conventional intuition of "more diverse is better"<n>We show that task diversity proves more critical than per-task demonstration quantity, benefiting transfer from diverse pre-training tasks to novel downstream scenarios.<n>We propose a distribution debiasing method to mitigate velocity ambiguity, the yielding GO-1-Pro achieves substantial performance gains of 15%, equivalent to using 2.5 times pre-training data.
  arXiv  Detail & Related papers  (2025-07-08T17:52:44Z)
• Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids [56.892520712892804]
  We introduce a practical sim-to-real RL recipe that trains a humanoid robot to perform three dexterous manipulation tasks.<n>We demonstrate high success rates on unseen objects and robust, adaptive policy behaviors.
  arXiv  Detail & Related papers  (2025-02-27T18:59:52Z)
• Human-Humanoid Robots Cross-Embodiment Behavior-Skill Transfer Using Decomposed Adversarial Learning from Demonstration [9.42179962375058]
  We propose a transferable framework that reduces the data bottleneck by using a unified digital human model as a common prototype.<n>The model learns behavior primitives from human demonstrations through adversarial imitation, and complex robot structures are decomposed into functional components.<n>Our framework is validated on five humanoid robots with diverse configurations.
  arXiv  Detail & Related papers  (2024-12-19T18:41:45Z)
• One-Shot Imitation under Mismatched Execution [7.060120660671016]
  Human demonstrations are a powerful way to program robots to do long-horizon manipulation tasks.<n> translating these demonstrations into robot-executable actions presents significant challenges due to execution mismatches in movement styles and physical capabilities.<n>We propose RHyME, a novel framework that automatically pairs human and robot trajectories using sequence-level optimal transport cost functions.
  arXiv  Detail & Related papers  (2024-09-10T16:11:57Z)
• Mitigating the Human-Robot Domain Discrepancy in Visual Pre-training for Robotic Manipulation [16.809190349155525]
  We propose a novel adaptation paradigm that leverages readily available paired human-robot video data to bridge the domain gap.<n>Our method employs a human-robot contrastive alignment loss to align the semantics of human and robot videos, adapting pre-trained models to the robot domain in a parameter-efficient manner.
  arXiv  Detail & Related papers  (2024-06-20T11:57:46Z)
• Domain Adaptive Robotic Gesture Recognition with Unsupervised Kinematic-Visual Data Alignment [60.31418655784291]
  We propose a novel unsupervised domain adaptation framework which can simultaneously transfer multi-modality knowledge, i.e., both kinematic and visual data, from simulator to real robot. It remedies the domain gap with enhanced transferable features by using temporal cues in videos, and inherent correlations in multi-modal towards recognizing gesture. Results show that our approach recovers the performance with great improvement gains, up to 12.91% in ACC and 20.16% in F1score without using any annotations in real robot.
  arXiv  Detail & Related papers  (2021-03-06T09:10:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.