Learning Diverse Humanoid Tasks via Synthetic Video Scenarios without Real World Data

• URL: http://arxiv.org/abs/2607.21648v1
• Date: Wed, 22 Jul 2026 05:36:55 GMT
• Title: Learning Diverse Humanoid Tasks via Synthetic Video Scenarios without Real World Data
• Abstract summary: We propose a new framework that leverages the power of Generative AI to convert textual prompts into realistic and diverse sequences of human body movements.<n>These synthetic demonstrations are then used as a training resource, allowing the robot to learn a broad range of task-execution styles.<n> Experimental results show that the robot not only completes the tasks successfully but also demonstrates strong adaptability to complex variations in motion.
• Score: 0.5161531917413708
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The human-like morphology of humanoid robots grants them exceptional potential for agile and versatile motor capabilities, but it also introduces significant challenges in acquiring complex skills. Traditional Learning-from-Demonstrations methods are often constrained by the high cost of collecting real-world data, the difficulty of capturing motion-specific behaviors, and the limited diversity of demonstrations across individuals. Moreover, even for the same task, humans may execute the motion in multiple distinct ways. In this paper, we propose a new framework that leverages the power of Generative AI to convert textual prompts into realistic and diverse sequences of human body movements, enabling the robot to observe multiple variations of how a single task can be performed. These synthetic demonstrations are then used as a training resource, allowing the robot to learn a broad range of task-execution styles without requiring direct human intervention. We evaluate the proposed method across four simulation scenarios. Experimental results show that the robot not only completes the tasks successfully but also demonstrates strong adaptability to complex variations in motion.

Related papers

• Practice Makes Policies: Bootstrapping and Consolidating Robotic Capabilities from Zero Human Demonstrations [9.01493948952274]
  General-purpose robotic manipulation requires robots to perform diverse tasks in open-world environments.<n> HERO is a self-improving hierarchical embodied agent that enables autonomous capability evolution from zero human demonstrations.
  arXiv  Detail & Related papers  (2026-07-29T11:51:41Z)
• Emergence of Human to Robot Transfer in Vision-Language-Action Models [88.76648919814771]
  Vision-language-action (VLA) models can enable broad open world generalization, but require large and diverse datasets.<n>We show that human-to-robot transfer emerges once the VLA is pre-trained on sufficient scenes, tasks, and embodiments.
  arXiv  Detail & Related papers  (2025-12-27T00:13:11Z)
• UniPrototype: Humn-Robot Skill Learning with Uniform Prototypes [4.338344229716167]
  UniPrototype is a novel framework that enables effective knowledge transfer from human to robot domains via shared motion primitives.<n>Our results show that UniPrototype successfully transfers human manipulation knowledge to robots, significantly improving learning efficiency and task performance.
  arXiv  Detail & Related papers  (2025-09-27T00:33:39Z)
• Interactive Imitation Learning for Dexterous Robotic Manipulation: Challenges and Perspectives -- A Survey [0.8287206589886879]
  Dexterous manipulation is a crucial yet highly complex challenge in humanoid robotics.<n>Survey reviews existing learning-based methods for real-world dexterous manipulation.<n>A promising yet underexplored direction is interactive imitation learning, where human feedback actively refines a robots behavior.
  arXiv  Detail & Related papers  (2025-05-30T12:19:32Z)
• Human-Humanoid Robots Cross-Embodiment Behavior-Skill Transfer Using Decomposed Adversarial Learning from Demonstration [9.42179962375058]
  We propose a transferable framework that reduces the data bottleneck by using a unified digital human model as a common prototype.<n>The model learns behavior primitives from human demonstrations through adversarial imitation, and complex robot structures are decomposed into functional components.<n>Our framework is validated on five humanoid robots with diverse configurations.
  arXiv  Detail & Related papers  (2024-12-19T18:41:45Z)
• VITAL: Interactive Few-Shot Imitation Learning via Visual Human-in-the-Loop Corrections [10.49712834719005]
  Imitation Learning (IL) has emerged as a powerful approach in robotics, allowing robots to acquire new skills by mimicking human actions.<n>Despite its potential, the data collection process for IL remains a significant challenge due to logistical difficulties and high costs associated with obtaining high-quality demonstrations.<n>We propose a large-scale data generation from a handful of demonstrations through data augmentation in simulation.
  arXiv  Detail & Related papers  (2024-07-30T23:29:47Z)
• HERD: Continuous Human-to-Robot Evolution for Learning from Human Demonstration [57.045140028275036]
  We show that manipulation skills can be transferred from a human to a robot through the use of micro-evolutionary reinforcement learning. We propose an algorithm for multi-dimensional evolution path searching that allows joint optimization of both the robot evolution path and the policy.
  arXiv  Detail & Related papers  (2022-12-08T15:56:13Z)
• BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning [108.41464483878683]
  We study the problem of enabling a vision-based robotic manipulation system to generalize to novel tasks. We develop an interactive and flexible imitation learning system that can learn from both demonstrations and interventions. When scaling data collection on a real robot to more than 100 distinct tasks, we find that this system can perform 24 unseen manipulation tasks with an average success rate of 44%.
  arXiv  Detail & Related papers  (2022-02-04T07:30:48Z)
• Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation [55.31301153979621]
  We tackle real-world long-horizon robot manipulation tasks through skill discovery. We present a bottom-up approach to learning a library of reusable skills from unsegmented demonstrations. Our method has shown superior performance over state-of-the-art imitation learning methods in multi-stage manipulation tasks.
  arXiv  Detail & Related papers  (2021-09-28T16:18:54Z)
• Lifelong Robotic Reinforcement Learning by Retaining Experiences [61.79346922421323]
  Many multi-task reinforcement learning efforts assume the robot can collect data from all tasks at all times. In this work, we study a practical sequential multi-task RL problem motivated by the practical constraints of physical robotic systems. We derive an approach that effectively leverages the data and policies learned for previous tasks to cumulatively grow the robot's skill-set.
  arXiv  Detail & Related papers  (2021-09-19T18:00:51Z)
• What Matters in Learning from Offline Human Demonstrations for Robot Manipulation [64.43440450794495]
  We conduct an extensive study of six offline learning algorithms for robot manipulation. Our study analyzes the most critical challenges when learning from offline human data. We highlight opportunities for learning from human datasets.
  arXiv  Detail & Related papers  (2021-08-06T20:48:30Z)
• SKID RAW: Skill Discovery from Raw Trajectories [23.871402375721285]
  It is desirable to only demonstrate full task executions instead of all individual skills. We propose a novel approach that simultaneously learns to segment trajectories into reoccurring patterns. The approach learns a skill conditioning that can be used to understand possible sequences of skills.
  arXiv  Detail & Related papers  (2021-03-26T17:27:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.