Human-like autonomy emerges from self-play and a pinch of human data

• URL: http://arxiv.org/abs/2606.19370v1
• Date: Thu, 11 Jun 2026 19:16:53 GMT
• Title: Human-like autonomy emerges from self-play and a pinch of human data
• Abstract summary: Self-play reinforcement learning has emerged as a way to train driving policies without any human data.<n>Our method treats human demonstrations as a regularization objective on top of a minimal safe goal-reaching reward.<n>Resulting policies coordinate with held-out human trajectories and complete training in 15 hours on a single consumer-grade GPU.
• Score: 5.286205985381964
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Self-play reinforcement learning has recently emerged as a way to train driving policies without any human data. It uses cheap, large-scale simulations to substitute expensive, large-scale human driving demonstrations. A key limitation of this approach is that policies trained through pure self-play can learn effective but alien driving conventions incompatible with people. Previous works attempt to mitigate such behavioral misalignments through extensive reward engineering and domain randomization, which are brittle and labor-intensive. Instead of completely discarding human demonstrations, our method treats them as a regularization objective on top of a minimal safe goal-reaching reward. Like the spice in a good stew, we find that a little human data goes a long way: our method uses only 30 minutes of human demonstrations, 2500x fewer than comparable imitation learning approaches. Resulting policies coordinate with held-out human trajectories and complete training in 15 hours on a single consumer-grade GPU. Videos and full source code are available at https://spiced-self-play.com/.

Related papers

• EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data [114.89243396877453]
  We present EgoScale, a human to dexterous manipulation transfer framework built on large scale egocentric human data.<n>We introduce a simple two stage transfer recipe: large scale human pretraining followed by lightweight aligned human robot mid training.<n>Our final policy improves average success rate by 54% over a no pretraining baseline using a 22 DoF dexterous robotic hand.
  arXiv  Detail & Related papers  (2026-02-18T18:59:05Z)
• Crossing the Human-Robot Embodiment Gap with Sim-to-Real RL using One Human Demonstration [21.94699075066712]
  We propose a novel real-to-sim-to-real framework for training dexterous manipulation policies using only one RGB-D video of a human demonstrating a task.<n>Human2Sim2Robot outperforms object-aware replay by over 55% and imitation learning by over 68% on grasping, non-prehensile manipulation, and multi-step tasks.
  arXiv  Detail & Related papers  (2025-04-17T03:15:20Z)
• Robust Autonomy Emerges from Self-Play [63.44745854476206]
  We show that robust and naturalistic driving emerges entirely from self-play in simulation at unprecedented scale.<n>This is enabled by Gigaflow, a batched simulator that can synthesize and train on 42 years of subjective driving experience per hour.
  arXiv  Detail & Related papers  (2025-02-05T16:41:05Z)
• One-Shot Imitation under Mismatched Execution [7.060120660671016]
  Human demonstrations are a powerful way to program robots to do long-horizon manipulation tasks.<n> translating these demonstrations into robot-executable actions presents significant challenges due to execution mismatches in movement styles and physical capabilities.<n>We propose RHyME, a novel framework that automatically pairs human and robot trajectories using sequence-level optimal transport cost functions.
  arXiv  Detail & Related papers  (2024-09-10T16:11:57Z)
• HumanPlus: Humanoid Shadowing and Imitation from Humans [82.47551890765202]
  We introduce a full-stack system for humanoids to learn motion and autonomous skills from human data. We first train a low-level policy in simulation via reinforcement learning using existing 40-hour human motion datasets. We then perform supervised behavior cloning to train skill policies using egocentric vision, allowing humanoids to complete different tasks autonomously.
  arXiv  Detail & Related papers  (2024-06-15T00:41:34Z)
• Learning a Universal Human Prior for Dexterous Manipulation from Human Preference [35.54663426598218]
  We propose a framework that learns a universal human prior using direct human preference feedback over videos. A task-agnostic reward model is trained through iteratively generating diverse polices and collecting human preference over the trajectories. Our method empirically demonstrates more human-like behaviors on robot hands in diverse tasks including even unseen tasks.
  arXiv  Detail & Related papers  (2023-04-10T14:17:33Z)
• Learning Reward Functions for Robotic Manipulation by Observing Humans [92.30657414416527]
  We use unlabeled videos of humans solving a wide range of manipulation tasks to learn a task-agnostic reward function for robotic manipulation policies. The learned rewards are based on distances to a goal in an embedding space learned using a time-contrastive objective.
  arXiv  Detail & Related papers  (2022-11-16T16:26:48Z)
• Minimizing Human Assistance: Augmenting a Single Demonstration for Deep Reinforcement Learning [0.0]
  We use a single human example collected through a simple-to-use virtual reality simulation to assist with RL training. Our method augments a single demonstration to generate numerous human-like demonstrations. Despite learning from a human example, the agent is not constrained to human-level performance.
  arXiv  Detail & Related papers  (2022-09-22T19:04:43Z)
• Human-to-Robot Imitation in the Wild [50.49660984318492]
  We propose an efficient one-shot robot learning algorithm, centered around learning from a third-person perspective. We show one-shot generalization and success in real-world settings, including 20 different manipulation tasks in the wild.
  arXiv  Detail & Related papers  (2022-07-19T17:59:59Z)
• Learning Accurate and Human-Like Driving using Semantic Maps and Attention [152.48143666881418]
  This paper investigates how end-to-end driving models can be improved to drive more accurately and human-like. We exploit semantic and visual maps from HERE Technologies and augment the existing Drive360 dataset with such. Our models are trained and evaluated on the Drive360 + HERE dataset, which features 60 hours and 3000 km of real-world driving data.
  arXiv  Detail & Related papers  (2020-07-10T22:25:27Z)
• Learning to Play by Imitating Humans [8.209859328381269]
  We show that it is possible to acquire a diverse set of skills by self-supervising control on top of human teleoperated play data. By training a behavioral cloning policy on a relatively small quantity of human play, we autonomously generate a large quantity of cloned play data. We demonstrate that a general purpose goal-conditioned policy trained on this augmented dataset substantially outperforms one trained only with the original human data.
  arXiv  Detail & Related papers  (2020-06-11T23:28:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.