RGB: RL Guided Whole-Body MPPI for Humanoid Control

• URL: http://arxiv.org/abs/2606.25123v1
• Date: Tue, 23 Jun 2026 19:54:02 GMT
• Title: RGB: RL Guided Whole-Body MPPI for Humanoid Control
• Abstract summary: We propose an RL guided whole-body model predictive path integral (MPPI) framework that acts as an add-on feedback controller.<n>The proposed method improves task-level precision over a pure RL baseline under the same command interface.
• Score: 2.854451361373021
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humanoid robots require whole-body controllers that are both robust and precise in contact-rich environments. While deep reinforcement learning (RL) achieves robust stability, its behavior is tightly coupled to the training objective and command interface, making it difficult to add new feedback objectives without retraining. In this study, we propose an RL guided whole-body model predictive path integral (MPPI) framework that acts as an add-on feedback controller on top of a pretrained RL policy. Instead of using RL policy as the final controller, we use it as a sampling prior that biases MPPI rollouts toward dynamically feasible behaviors. Task objectives are specified through modular MPPI cost terms, and MPPI closes the loop by continuously correcting the RL prior online to satisfy these objectives without retraining the policy. Simulations on a 29-DoF Unitree G1 humanoid in MuJoCo demonstrate stable high-rate control (average 280~Hz). The proposed method improves task-level precision over a pure RL baseline under the same command interface. This is achieved by correcting systematic drift during straight walking and tracking additional whole-body reference signals imposed through the cost.

Related papers

• SRL-MPC: Shape-Aware Reinforcement Learned Model Predictive Control [64.66285242901918]
  Shape-Aware Reinforcement Learned Model Predictive Control (SRL-MPC) is a method for safe, efficient, and adaptive navigation in crowds with heterogeneous shapes without geometry simplification.<n>A reinforcement learning framework then learns a neural policy that reads geometric separation features (GSFs) and outputs real-time MPC parameter updates.<n> Experiments in randomized crowd scenarios with arbitrary shaped robot fleets demonstrate the effectiveness, scalability, and robustness of SRL-MPC.
  arXiv  Detail & Related papers  (2026-08-21T14:46:06Z)
• Deep Reinforcement Learning for Spacecraft Attitude Control During Atmospheric Re-Entry [37.22137915015052]
  We explore reinforcement learning for attitude control in spacecraft re-entry.<n>We formalize the application in the RL framework to apply continuous, off-policy RL.<n>We show that hybrid controllers are able to track the angle of attack better and are more robust under variations of mass, inertia tensor, and flap actuator bandwidth.
  arXiv  Detail & Related papers  (2026-06-30T08:06:07Z)
• MPC-Injection: Biasing Off-Policy Locomotion RL Toward Controller-Induced Behavior Basins [7.729687505744033]
  We present MPC-Injection, a low-overhead method that drives the policy into the controller's behavior basin.<n>Unlike reward shaping, MPC-Injection does not require redesigning the task reward.<n>We show that MPC-Injection drives the policy into the controller's behavior basin using a one to two-term task reward.
  arXiv  Detail & Related papers  (2026-06-24T21:23:22Z)
• RLoop: An Self-Improving Framework for Reinforcement Learning with Iterative Policy Initialization [65.23034604711489]
  We introduce RLoop, a self-improving framework for training large reasoning models.<n>RLoop transforms the standard training process into a virtuous cycle: it first uses RL to explore the solution space from a given policy, then filters the successful trajectories to create an expert dataset.<n>Our experiments show RLoops forgetting and substantially improves generalization, boosting average accuracy by 9% and pass@32 by over 15% compared to vanilla RL.
  arXiv  Detail & Related papers  (2025-11-06T11:27:16Z)
• M$^3$PC: Test-time Model Predictive Control for Pretrained Masked Trajectory Model [14.779390462893298]
  We propose using Model Predictive Control (MPC) at test time to leverage the model's own predictive capability to guide its action selection.<n>MPC significantly improves the decision-making performance of a pretrained trajectory model without any additional parameter training.<n>Our framework can be adapted to Offline to Online (O2O) RL and Goal Reaching RL.
  arXiv  Detail & Related papers  (2024-12-07T14:44:22Z)
• DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning [61.10299147201369]
  This paper introduces a novel autonomous RL approach, called DigiRL, for training in-the-wild device control agents. We build a scalable and parallelizable Android learning environment equipped with a VLM-based evaluator. We demonstrate the effectiveness of DigiRL using the Android-in-the-Wild dataset, where our 1.3B VLM trained with RL achieves a 49.5% absolute improvement.
  arXiv  Detail & Related papers  (2024-06-14T17:49:55Z)
• REBEL: Reward Regularization-Based Approach for Robotic Reinforcement Learning from Human Feedback [61.54791065013767]
  A misalignment between the reward function and human preferences can lead to catastrophic outcomes in the real world.<n>Recent methods aim to mitigate misalignment by learning reward functions from human preferences.<n>We propose a novel concept of reward regularization within the robotic RLHF framework.
  arXiv  Detail & Related papers  (2023-12-22T04:56:37Z)
• Train Hard, Fight Easy: Robust Meta Reinforcement Learning [78.16589993684698]
  A major challenge of reinforcement learning (RL) in real-world applications is the variation between environments, tasks or clients. Standard MRL methods optimize the average return over tasks, but often suffer from poor results in tasks of high risk or difficulty. In this work, we define a robust MRL objective with a controlled level. The data inefficiency is addressed via the novel Robust Meta RL algorithm (RoML)
  arXiv  Detail & Related papers  (2023-01-26T14:54:39Z)
• Robust Deep Reinforcement Learning for Quadcopter Control [0.8687092759073857]
  In this work, we use Robust Markov Decision Processes (RMDP) to train the drone control policy. It opts for pessimistic optimization to handle potential gaps between policy transfer from one environment to another. The trained control policy is tested on the task of quadcopter positional control.
  arXiv  Detail & Related papers  (2021-11-06T16:35:13Z)
• Pareto Deterministic Policy Gradients and Its Application in 5G Massive MIMO Networks [32.099949375036495]
  We consider jointly optimizing cell load balance and network throughput via a reinforcement learning (RL) approach. Our rationale behind using RL is to circumvent the challenges of analytically modeling user mobility and network dynamics. To accomplish this joint optimization, we integrate vector rewards into the RL value network and conduct RL action via a separate policy network.
  arXiv  Detail & Related papers  (2020-12-02T15:35:35Z)
• Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion [78.46388769788405]
  We introduce guided constrained policy optimization (GCPO), an RL framework based upon our implementation of constrained policy optimization (CPPO) We show that guided constrained RL offers faster convergence close to the desired optimum resulting in an optimal, yet physically feasible, robotic control behavior without the need for precise reward function tuning.
  arXiv  Detail & Related papers  (2020-02-22T10:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.