Related papers: Learning Sim-to-Real Humanoid Locomotion in 15 Minutes

Learning Sim-to-Real Humanoid Locomotion in 15 Minutes

URL: http://arxiv.org/abs/2512.01996v1
Date: Mon, 01 Dec 2025 18:55:17 GMT
Title: Learning Sim-to-Real Humanoid Locomotion in 15 Minutes
Authors: Younggyo Seo, Carmelo Sferrazza, Juyue Chen, Guanya Shi, Rocky Duan, Pieter Abbeel,
Abstract summary: We introduce a simple and practical recipe based on off-policy RL algorithms, i.e., FastSAC and FastTD3.<n>Our simple recipe stabilizes off-policy RL algorithms at massive scale with thousands of parallel environments.<n>We demonstrate rapid end-to-end learning of humanoid locomotion controllers on Unitree G1 and Booster T1 robots.
Score: 51.500643119683225
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Massively parallel simulation has reduced reinforcement learning (RL) training time for robots from days to minutes. However, achieving fast and reliable sim-to-real RL for humanoid control remains difficult due to the challenges introduced by factors such as high dimensionality and domain randomization. In this work, we introduce a simple and practical recipe based on off-policy RL algorithms, i.e., FastSAC and FastTD3, that enables rapid training of humanoid locomotion policies in just 15 minutes with a single RTX 4090 GPU. Our simple recipe stabilizes off-policy RL algorithms at massive scale with thousands of parallel environments through carefully tuned design choices and minimalist reward functions. We demonstrate rapid end-to-end learning of humanoid locomotion controllers on Unitree G1 and Booster T1 robots under strong domain randomization, e.g., randomized dynamics, rough terrain, and push perturbations, as well as fast training of whole-body human-motion tracking policies. We provide videos and open-source implementation at: https://younggyo.me/fastsac-humanoid.

Related papers

Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer [59.02729900344616]
GPU-accelerated, photorealistic simulation has opened a scalable data-generation path for robot learning.<n>We develop a teacher-student-bootstrap learning framework for vision-based humanoid loco-manipulation.<n>This represents the first humanoid sim-to-real policy capable of diverse articulated loco-manipulation using pure RGB perception.
arXiv Detail & Related papers (2025-11-30T20:07:13Z)
FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Control [49.08235196039602]
FastTD3 is aReinforcement Learning (RL) algorithm that solves a range of HumanoidBench tasks in under 3 hours on a single A100 GPU.<n>We also provide a lightweight and easy-to-use implementation of FastTD3 to accelerate RL research in robotics.
arXiv Detail & Related papers (2025-05-28T17:55:26Z)
Autonomous Vehicle Controllers From End-to-End Differentiable Simulation [57.278726604424556]
We propose a differentiable simulator and design an analytic policy gradients (APG) approach to training AV controllers.<n>Our proposed framework brings the differentiable simulator into an end-to-end training loop, where gradients of environment dynamics serve as a useful prior to help the agent learn a more grounded policy.<n>We find significant improvements in performance and robustness to noise in the dynamics, as well as overall more intuitive human-like handling.
arXiv Detail & Related papers (2024-09-12T11:50:06Z)
DrEureka: Language Model Guided Sim-To-Real Transfer [64.14314476811806]
Transferring policies learned in simulation to the real world is a promising strategy for acquiring robot skills at scale. In this paper, we investigate using Large Language Models (LLMs) to automate and accelerate sim-to-real design. Our approach is capable of solving novel robot tasks, such as quadruped balancing and walking atop a yoga ball.
arXiv Detail & Related papers (2024-06-04T04:53:05Z)
Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning [26.13655448415553]
Deep Reinforcement Learning (Deep RL) is able to synthesize sophisticated and safe movement skills for a low-cost, miniature humanoid robot. We used Deep RL to train a humanoid robot with 20 actuated joints to play a simplified one-versus-one (1v1) soccer game. The resulting agent exhibits robust and dynamic movement skills such as rapid fall recovery, walking, turning, kicking and more.
arXiv Detail & Related papers (2023-04-26T16:25:54Z)
Learning Bipedal Walking for Humanoids with Current Feedback [5.429166905724048]
We present an approach for overcoming the sim2real gap issue for humanoid robots arising from inaccurate torque-tracking at the actuator level. Our approach successfully trains a unified, end-to-end policy in simulation that can be deployed on a real HRP-5P humanoid robot to achieve bipedal locomotion.
arXiv Detail & Related papers (2023-03-07T08:16:46Z)
Rapid Locomotion via Reinforcement Learning [15.373208553045416]
We present an end-to-end learned controller that achieves record agility for the MIT Mini Cheetah. This system runs and turns fast on natural terrains like grass, ice, and gravel and responds robustly to disturbances.
arXiv Detail & Related papers (2022-05-05T17:55:11Z)
Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomization [9.014518402531875]
We train the RL policy for end-to-end control in a physical simulator without any predefined controller or reference motion. We demonstrate the robot can smoothly maneuver at a high speed, avoid the obstacles, and show significant improvement over the baselines.
arXiv Detail & Related papers (2021-09-29T16:48:05Z)
Learning to Walk in the Real World with Minimal Human Effort [80.7342153519654]
We develop a system for learning legged locomotion policies with deep RL in the real world with minimal human effort. Our system can automatically and efficiently learn locomotion skills on a Minitaur robot with little human intervention.
arXiv Detail & Related papers (2020-02-20T03:36:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.