Robot Self-Improvement via Human-Video Dynamics Models
Abstract Overview
This paper studies whether priors learned from passive human videos can support not only robot policy initialization but also autonomous self-improvement after deployment. The authors learn embodiment-agnostic action, dynamics, and value representations from human videos, then adapt the dynamics and value models with robot rollouts. They introduce Dynamics-Guided Action Correction (DGAC), a training-free mechanism that treats failed robot states as queries, generates candidate corrective actions, predicts their outcomes with a dynamics model, and ranks them with a value model. The system is evaluated on seven real-world manipulation tasks across two robot embodiments and multiple policy backbones.
Novelty
The distinctive contribution is using human videos to pretrain transferable action, dynamics, and value models that later support failure-driven robot self-improvement across embodiments. The proposed DGAC method is also novel in converting failed rollout states into corrective supervision offline, without online human intervention or additional training for the correction module.
Results
Across five main real-world tasks on Hello Robot Stretch 3, the full method achieves an average success rate of 85.3%, compared with 76.0% for the strongest reported baseline and 41.3% for the behavior-cloning initialization. Applied to a different backbone, DGAC improves a fine-tuned π0.5 policy from 62.7% to 88.0%, substantially exceeding RECAP without human correction at 68.0%. On Franka Panda, the method raises average success from 36.7% to 70.0% across two tasks, indicating cross-embodiment transfer.
Key Points
- Human-video pretraining is used to learn embodiment-agnostic policy, dynamics, and value representations that can be grounded with robot rollouts.
- DGAC repairs recoverable failures by generating locally adapted corrective actions from successful references and selecting the candidate with the highest predicted future value.
- Experiments show strong gains over prior self-improvement baselines, transfer to a different robot embodiment, and effectiveness on a separate pretrained policy backbone.