Scaling Single Human Demonstrations for Imitation Learning using Generative Foundational Models

• URL: http://arxiv.org/abs/2602.12734v1
• Date: Fri, 13 Feb 2026 09:04:18 GMT
• Title: Scaling Single Human Demonstrations for Imitation Learning using Generative Foundational Models
• Abstract summary: We propose Real2Gen to train a manipulation policy from a single human demonstration.<n>Real2Gen extracts required information from the demonstration and transfers it to a simulation environment.<n>We show an average increase in the success rate of 26.6% and better generalization of the trained policy.
• Score: 16.365940498738116
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Imitation learning is a popular paradigm to teach robots new tasks, but collecting robot demonstrations through teleoperation or kinesthetic teaching is tedious and time-consuming. In contrast, directly demonstrating a task using our human embodiment is much easier and data is available in abundance, yet transfer to the robot can be non-trivial. In this work, we propose Real2Gen to train a manipulation policy from a single human demonstration. Real2Gen extracts required information from the demonstration and transfers it to a simulation environment, where a programmable expert agent can demonstrate the task arbitrarily many times, generating an unlimited amount of data to train a flow matching policy. We evaluate Real2Gen on human demonstrations from three different real-world tasks and compare it to a recent baseline. Real2Gen shows an average increase in the success rate of 26.6% and better generalization of the trained policy due to the abundance and diversity of training data. We further deploy our purely simulation-trained policy zero-shot in the real world. We make the data, code, and trained models publicly available at real2gen.cs.uni-freiburg.de.

Related papers

• HumanoidMimicGen: Data Generation for Loco-Manipulation via Whole-Body Planning [52.022681285103126]
  We present HumanoidMimicGen, a method for generating humanoid legged loco-manipulation data.<n>Our method adapts contact-rich whole-body skills from a handful of source demonstrations to new states.<n>We show that whole-body visuomotor policies co-trained with data generated by HumanoidMimicGen outperform those trained only on real-world data by 20%.
  arXiv  Detail & Related papers  (2026-05-26T21:57:11Z)
• Crossing the Human-Robot Embodiment Gap with Sim-to-Real RL using One Human Demonstration [21.94699075066712]
  We propose a novel real-to-sim-to-real framework for training dexterous manipulation policies using only one RGB-D video of a human demonstrating a task.<n>Human2Sim2Robot outperforms object-aware replay by over 55% and imitation learning by over 68% on grasping, non-prehensile manipulation, and multi-step tasks.
  arXiv  Detail & Related papers  (2025-04-17T03:15:20Z)
• Video2Policy: Scaling up Manipulation Tasks in Simulation through Internet Videos [61.925837909969815]
  We introduce Video2Policy, a novel framework that leverages internet RGB videos to reconstruct tasks based on everyday human behavior.<n>Our method can successfully train RL policies on such tasks, including complex and challenging tasks such as throwing.<n>We show that the generated simulation data can be scaled up for training a general policy, and it can be transferred back to the real robot in a Real2Sim2Real way.
  arXiv  Detail & Related papers  (2025-02-14T03:22:03Z)
• Robot Learning with Super-Linear Scaling [13.053949644385932]
  CASHER is a pipeline for scaling up data collection and learning in simulation where the performance scales superlinearly with human effort.<n>We show that CASHER enables fine-tuning of pre-trained policies to a target scenario using a video scan without any additional human effort.
  arXiv  Detail & Related papers  (2024-12-02T18:12:02Z)
• Instant Policy: In-Context Imitation Learning via Graph Diffusion [12.879700241782528]
  In-context Imitation Learning (ICIL) is a promising opportunity for robotics.<n>We introduce Instant Policy, which learns new tasks instantly from just one or two demonstrations.<n>We also show how it can serve as a foundation for cross-embodiment and zero-shot transfer to language-defined tasks.
  arXiv  Detail & Related papers  (2024-11-19T16:45:52Z)
• Manipulate-Anything: Automating Real-World Robots using Vision-Language Models [47.16659229389889]
  We propose Manipulate-Anything, a scalable automated generation method for real-world robotic manipulation. Manipulate-Anything can operate in real-world environments without any privileged state information, hand-designed skills, and can manipulate any static object.
  arXiv  Detail & Related papers  (2024-06-27T06:12:01Z)
• MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations [55.549956643032836]
  MimicGen is a system for automatically synthesizing large-scale, rich datasets from only a small number of human demonstrations. We show that robot agents can be effectively trained on this generated dataset by imitation learning to achieve strong performance in long-horizon and high-precision tasks.
  arXiv  Detail & Related papers  (2023-10-26T17:17:31Z)
• Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning [58.3994826169858]
  We introduce RoboFuME, a reset-free fine-tuning system for robotic reinforcement learning. Our insights are to utilize offline reinforcement learning techniques to ensure efficient online fine-tuning of a pre-trained policy. Our method can incorporate data from an existing robot dataset and improve on a target task within as little as 3 hours of autonomous real-world experience.
  arXiv  Detail & Related papers  (2023-10-23T17:50:08Z)
• Learning Reward Functions for Robotic Manipulation by Observing Humans [92.30657414416527]
  We use unlabeled videos of humans solving a wide range of manipulation tasks to learn a task-agnostic reward function for robotic manipulation policies. The learned rewards are based on distances to a goal in an embedding space learned using a time-contrastive objective.
  arXiv  Detail & Related papers  (2022-11-16T16:26:48Z)
• A Framework for Efficient Robotic Manipulation [79.10407063260473]
  We show that a single robotic arm can learn sparse-reward manipulation policies from pixels. We show that, given only 10 demonstrations, a single robotic arm can learn sparse-reward manipulation policies from pixels.
  arXiv  Detail & Related papers  (2020-12-14T22:18:39Z)
• Visual Imitation Made Easy [102.36509665008732]
  We present an alternate interface for imitation that simplifies the data collection process while allowing for easy transfer to robots. We use commercially available reacher-grabber assistive tools both as a data collection device and as the robot's end-effector. We experimentally evaluate on two challenging tasks: non-prehensile pushing and prehensile stacking, with 1000 diverse demonstrations for each task.
  arXiv  Detail & Related papers  (2020-08-11T17:58:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.