RPL: Learning Robust Humanoid Perceptive Locomotion on Challenging Terrains

• URL: http://arxiv.org/abs/2602.03002v1
• Date: Tue, 03 Feb 2026 02:17:08 GMT
• Title: RPL: Learning Robust Humanoid Perceptive Locomotion on Challenging Terrains
• Abstract summary: We propose a two-stage training framework that enables multi-directional locomotion on challenging terrains.<n>RPL first trains expert policies with privileged height map observations to master decoupled locomotion and manipulation skills.<n>We then distill them into a transformer policy that leverages multiple depth cameras to cover a wide range of views.<n>For scalable depth distillation, we develop an efficient multi-depth system that ray-casts against both dynamic robot meshes and static terrain meshes.
• Score: 53.674116167774095
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humanoid perceptive locomotion has made significant progress and shows great promise, yet achieving robust multi-directional locomotion on complex terrains remains underexplored. To tackle this challenge, we propose RPL, a two-stage training framework that enables multi-directional locomotion on challenging terrains, and remains robust with payloads. RPL first trains terrain-specific expert policies with privileged height map observations to master decoupled locomotion and manipulation skills across different terrains, and then distills them into a transformer policy that leverages multiple depth cameras to cover a wide range of views. During distillation, we introduce two techniques to robustify multi-directional locomotion, depth feature scaling based on velocity commands and random side masking, which are critical for asymmetric depth observations and unseen widths of terrains. For scalable depth distillation, we develop an efficient multi-depth system that ray-casts against both dynamic robot meshes and static terrain meshes in massively parallel environments, achieving a 5-times speedup over the depth rendering pipelines in existing simulators while modeling realistic sensor latency, noise, and dropout. Extensive real-world experiments demonstrate robust multi-directional locomotion with payloads (2kg) across challenging terrains, including 20° slopes, staircases with different step lengths (22 cm, 25 cm, 30 cm), and 25 cm by 25 cm stepping stones separated by 60 cm gaps.

Related papers

• Agile perceptive multi-skill locomotion for quadrupedal robots in the wild [4.14240816913171]
  APT-RL (Action Pretrained Transformer-based Reinforcement Learning) is a unified framework that enables multi-skill locomotion.<n>Our approach generates large-scale, feature-rich 2D motion datasets through trajectory optimization with simplified dynamics.<n>The resulting high-quality skills serve as strong priors for efficient learning of complex downstream tasks and extend naturally to 3D environments.
  arXiv  Detail & Related papers  (2026-07-15T08:22:52Z)
• CReF: Cross-modal and Recurrent Fusion for Depth-conditioned Humanoid Locomotion [6.2747272941647045]
  We propose CReF (Cross-modal and Recurrent Fusion), a single-stage depth-conditioned humanoid locomotion framework.<n>CReF learns locomotion-relevant features directly from raw forward-facing depth without explicit geometric intermediates.
  arXiv  Detail & Related papers  (2026-03-31T08:56:29Z)
• TerraSkipper: A Centimeter-Scale Robot for Multi-Terrain Skipping and Crawling [2.683233968306505]
  Mudskippers are unique amphibious fish capable of locomotion in diverse environments.<n>We present the design and development of a centimeter-scale skipping and crawling robot.<n>It is equipped with two side fins for crawling, each integrated with a hall effect sensor for gait control.<n>A rotary springtail driven by a 10mm planetary gear motor enables continuous impulsive skipping across a range of substrates.
  arXiv  Detail & Related papers  (2026-03-29T14:59:42Z)
• APEX: Learning Adaptive High-Platform Traversal for Humanoid Robots [32.84976822343718]
  We propose APEX, a system for perceptive, climbing-based high-platform traversal behaviors.<n>Central to our approach is a generalized ratchet progress reward for learning contact-rich, goal-reaching maneuvers.<n>We train LiDAR-based full-body maneuver policies and reduce the sim-to-real perception gap.
  arXiv  Detail & Related papers  (2026-02-11T18:55:11Z)
• RoboTracer: Mastering Spatial Trace with Reasoning in Vision-Language Models for Robotics [53.053660003572965]
  We propose RoboTracer, a 3D-aware VLM that first achieves both 3D spatial referring and measuring.<n>RoboTracer advances multi-step metric-grounded reasoning via reinforcement fine-tuning.<n>We present TraceSpatial-Bench, a challenging benchmark to evaluate spatial tracing.
  arXiv  Detail & Related papers  (2025-12-15T18:52:43Z)
• Gait-Adaptive Perceptive Humanoid Locomotion with Real-Time Under-Base Terrain Reconstruction [86.53248703859718]
  We introduce a perceptive locomotion framework that merges terrain sensing, gait regulation, and whole-body control into a single reinforcement learning policy.<n>Experiments conducted on a 31-DoF, 1.65 m humanoid robot demonstrate robust locomotion in both simulation and real-world settings.
  arXiv  Detail & Related papers  (2025-12-08T11:42:41Z)
• Learning to Track Any Points from Human Motion [55.831218129679144]
  We propose an automated pipeline to generate pseudo-labeled training data for point tracking.<n>A point tracking model trained on AnthroTAP achieves annotated state-of-the-art performance on the TAP-Vid benchmark.
  arXiv  Detail & Related papers  (2025-07-08T17:59:58Z)
• Scaling Multi Agent Reinforcement Learning for Underwater Acoustic Tracking via Autonomous Vehicles [1.7923169244369488]
  Multi-Agent Reinforcement Learning is notoriously sample-inefficient.<n>High-fidelity simulators like Gazebo's LRAUV provide 100x faster-than-real-time single-robot simulations.<n>We propose an iterative distillation method that transfers high-fidelity simulations into a simplified, GPU-accelerated environment.
  arXiv  Detail & Related papers  (2025-05-13T04:42:30Z)
• BeamDojo: Learning Agile Humanoid Locomotion on Sparse Footholds [35.62230804783507]
  Existing learning-based approaches often struggle on complex terrains due to sparse foothold rewards and inefficient learning processes.<n>We introduce BeamDojo, a reinforcement learning framework designed for enabling agile humanoid locomotion on sparse footholds.<n>We show that BeamDojo achieves efficient learning in simulation and enables agile locomotion with precise foot placement on sparse footholds in the real world.
  arXiv  Detail & Related papers  (2025-02-14T18:42:42Z)
• Legged Locomotion in Challenging Terrains using Egocentric Vision [70.37554680771322]
  We present the first end-to-end locomotion system capable of traversing stairs, curbs, stepping stones, and gaps. We show this result on a medium-sized quadruped robot using a single front-facing depth camera.
  arXiv  Detail & Related papers  (2022-11-14T18:59:58Z)
• VAE-Loco: Versatile Quadruped Locomotion by Learning a Disentangled Gait Representation [78.92147339883137]
  We show that it is pivotal in increasing controller robustness by learning a latent space capturing the key stance phases constituting a particular gait. We demonstrate that specific properties of the drive signal map directly to gait parameters such as cadence, footstep height and full stance duration. The use of a generative model facilitates the detection and mitigation of disturbances to provide a versatile and robust planning framework.
  arXiv  Detail & Related papers  (2022-05-02T19:49:53Z)
• Learning Perceptual Locomotion on Uneven Terrains using Sparse Visual Observations [75.60524561611008]
  This work aims to exploit the use of sparse visual observations to achieve perceptual locomotion over a range of commonly seen bumps, ramps, and stairs in human-centred environments. We first formulate the selection of minimal visual input that can represent the uneven surfaces of interest, and propose a learning framework that integrates such exteroceptive and proprioceptive data. We validate the learned policy in tasks that require omnidirectional walking over flat ground and forward locomotion over terrains with obstacles, showing a high success rate.
  arXiv  Detail & Related papers  (2021-09-28T20:25:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.