MARCH: Model-Assisted Reinforcement Learning for the Perceptive Control of Humanoids over Sparse Footholds

• URL: http://arxiv.org/abs/2606.10288v1
• Date: Tue, 09 Jun 2026 01:21:26 GMT
• Title: MARCH: Model-Assisted Reinforcement Learning for the Perceptive Control of Humanoids over Sparse Footholds
• Abstract summary: We propose a model-assisted reinforcement learning (RL) framework that combines both perspectives in three steps.<n>We show that this model-assistance procedure produces physically grounded locomotion, improving sample efficiency.<n>We demonstrate successful deployment on a Unitree G1 humanoid robot navigating sparse footholds with lateral constraints.
• Score: 2.3940819037450987
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Perceptive bipedal locomotion over sparse terrain remains a difficult challenge: model-based methods are precise but brittle to uncertainty, while model-free methods are robust but struggle to discover the precise, constrained motions required for safety-critical locomotion where small errors can cause catastrophic failures. We propose a model-assisted reinforcement learning (RL) framework that combines both perspectives in three steps: (1) generate a safe reference trajectory using simplified models; (2) train a privileged teacher policy guided by a control Lyapunov function (CLF) reward built around the safe reference trajectory; and (3) distill the teacher into a vision-based student policy. We show that this model-assistance procedure produces physically grounded locomotion, improving sample efficiency, reducing the need for a complex learning curriculum, and achieving smoother locomotion behavior alongside stepping stone performance comparable to model-free baselines. We validate our approach in simulation and demonstrate successful deployment on a Unitree G1 humanoid robot navigating sparse footholds with lateral constraints.

Related papers

• Efficiently Learning Robust Torque-based Locomotion Through Reinforcement with Model-Based Supervision [14.246416873754905]
  We propose a control framework that integrates model-based bipedal locomotion with residual reinforcement learning.<n>Our method demonstrates improved robustness and generalization across a range of randomized conditions.
  arXiv  Detail & Related papers  (2026-01-22T16:56:52Z)
• Walk the PLANC: Physics-Guided RL for Agile Humanoid Locomotion on Constrained Footholds [20.44610499489492]
  Bipedal humanoid robots must coordinate balance, timing, and contact decisions when locomoting on constrained footholds.<n>We introduce a locomotion framework in which a reduced-order stepping planner supplies dynamically consistent motion targets.<n>This combination of structured footstep planning and data-driven adaptation produces accurate, agile, and hardware-validated stepping-stone locomotion on a humanoid robot.
  arXiv  Detail & Related papers  (2026-01-09T19:56:42Z)
• Dynamic Policy Learning for Legged Robot with Simplified Model Pretraining and Model Homotopy Transfer [9.720210672100762]
  We introduce a continuation-based learning framework that combines model pretraining and model homotopy transfer to efficiently generate and refine complex dynamic behaviors.<n>Our framework is validated on a range of dynamic tasks, including flips and wall-assisted maneuvers, and is successfully deployed on a realpedalru robot.
  arXiv  Detail & Related papers  (2025-12-31T08:04:22Z)
• Human-in-the-loop Online Rejection Sampling for Robotic Manipulation [55.99788088622936]
  Hi-ORS stabilizes value estimation by filtering out negatively rewarded samples during online fine-tuning.<n>Hi-ORS fine-tunes a pi-base policy to master contact-rich manipulation in just 1.5 hours of real-world training.
  arXiv  Detail & Related papers  (2025-10-30T11:53:08Z)
• Action Flow Matching for Continual Robot Learning [54.10050120844738]
  Continual learning in robotics seeks systems that can constantly adapt to changing environments and tasks.<n>We introduce a generative framework leveraging flow matching for online robot dynamics model alignment.<n>We find that by transforming the actions themselves rather than exploring with a misaligned model, the robot collects informative data more efficiently.
  arXiv  Detail & Related papers  (2025-04-25T16:26:15Z)
• Dynamic Obstacle Avoidance with Bounded Rationality Adversarial Reinforcement Learning [5.760394464143113]
  We propose a novel way to endow navigation policies with robustness by a training process that models obstacles as adversarial agents.<n>We call this method versa policies via Quantal response Adrial Reinforcement Learning (Hi-QARL)
  arXiv  Detail & Related papers  (2025-03-14T14:54:02Z)
• Differentiable Information Enhanced Model-Based Reinforcement Learning [48.820039382764]
  Differentiable environments have heralded new possibilities for learning control policies by offering rich differentiable information.<n>Model-based reinforcement learning (MBRL) methods exhibit the potential to effectively harness the power of differentiable information for recovering the underlying physical dynamics.<n>However, this presents two primary challenges: effectively utilizing differentiable information to 1) construct models with more accurate dynamic prediction and 2) enhance the stability of policy training.
  arXiv  Detail & Related papers  (2025-03-03T04:51:40Z)
• DTC: Deep Tracking Control [16.2850135844455]
  We propose a hybrid control architecture that combines the advantages of both worlds to achieve greater robustness, foot-placement accuracy, and terrain generalization. A deep neural network policy is trained in simulation, aiming to track the optimized footholds. We demonstrate superior robustness in the presence of slippery or deformable ground when compared to model-based counterparts.
  arXiv  Detail & Related papers  (2023-09-27T07:57:37Z)
• Training and Evaluation of Deep Policies using Reinforcement Learning and Generative Models [67.78935378952146]
  GenRL is a framework for solving sequential decision-making problems. It exploits the combination of reinforcement learning and latent variable generative models. We experimentally determine the characteristics of generative models that have most influence on the performance of the final policy training.
  arXiv  Detail & Related papers  (2022-04-18T22:02:32Z)
• An Adaptable Approach to Learn Realistic Legged Locomotion without Examples [38.81854337592694]
  This work proposes a generic approach for ensuring realism in locomotion by guiding the learning process with the spring-loaded inverted pendulum model as a reference. We present experimental results showing that even in a model-free setup, the learned policies can generate realistic and energy-efficient locomotion gaits for a bipedal and a quadrupedal robot.
  arXiv  Detail & Related papers  (2021-10-28T10:14:47Z)
• Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots [121.42930679076574]
  We present a model-free reinforcement learning framework for training robust locomotion policies in simulation. domain randomization is used to encourage the policies to learn behaviors that are robust across variations in system dynamics. We demonstrate this on versatile walking behaviors such as tracking a target walking velocity, walking height, and turning yaw.
  arXiv  Detail & Related papers  (2021-03-26T07:14:01Z)
• Bridging Imagination and Reality for Model-Based Deep Reinforcement Learning [72.18725551199842]
  We propose a novel model-based reinforcement learning algorithm, called BrIdging Reality and Dream (BIRD) It maximizes the mutual information between imaginary and real trajectories so that the policy improvement learned from imaginary trajectories can be easily generalized to real trajectories. We demonstrate that our approach improves sample efficiency of model-based planning, and achieves state-of-the-art performance on challenging visual control benchmarks.
  arXiv  Detail & Related papers  (2020-10-23T03:22:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.