Steering Generative Reinforcement Learning into Stable Robotic Controller

• URL: http://arxiv.org/abs/2606.16572v1
• Date: Mon, 15 Jun 2026 11:16:37 GMT
• Title: Steering Generative Reinforcement Learning into Stable Robotic Controller
• Abstract summary: SteerGenPO is a latent-space reinforcement learning framework that steers a trained generative policy into a robust deterministic robotic controller.<n>We evaluate SteerGenPO on six Isaac Lab benchmarks and a Unitree G1 locomotion task.
• Score: 40.70263465163786
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion and flow-based generative policies provide a powerful policy class for reinforcement learning by inducing rich stochastic exploration through iterative action generation. However, the stochasticity of diffusion policies is not suitable for stable and precise control in high-dimensional robotic systems, where small action variations can accumulate into inconsistent motion and reduced robustness. To address this issue, we propose SteerGenPO, a latent-space reinforcement learning framework that steers a trained generative policy into a robust deterministic robotic controller. The key idea is to replace stochastic latent sampling of the trained generative policy with a learned latent actor that predicts a state-dependent latent input for the generative policies. This separates exploration and control: stochastic generative sampling provides diverse action proposals during policy learning, while deterministic latent steering provides stable and adaptive control at deployment. We evaluate SteerGenPO on six Isaac Lab benchmarks and a Unitree G1 locomotion task. The results show SteerGenPO improves over both classical RL and generative RL baselines, while its deterministic latent steering produces more stable inference-time behaviors and more reliable command responses.

Related papers

• Beyond Noise Steering: Dual-Latent Space Reinforcement Learning for Generative Robot Policy [13.530890681678619]
  Pretrained generative robot policies learn expressive action prior to demonstrations.<n>Existing reinforcement learning methods only steer the noisy space but fail to modulate intermediate action representations during the generation process.<n>We propose a novel Dual-Latent Space Reinforcement Learning framework, which complements initial-noise steering with representation-level control inside the frozen generator.
  arXiv  Detail & Related papers  (2026-09-10T09:04:04Z)
• Deep Reinforcement Learning for Robotic Manipulation under Distribution Shift with Bounded Extremum Seeking [42.278434352794676]
  Reinforcement learning has shown strong performance in robotic manipulation, but learned policies often degrade in performance when test conditions differ from the training distribution.<n>This limitation is especially important in contact-rich tasks such as pushing and pick-and-place, where changes in goals, contact conditions, or robot dynamics can drive the system out-of-distribution at inference time.<n>In this paper, we investigate a hybrid controller that combines reinforcement learning with bounded extremum seeking to improve robustness under such conditions.
  arXiv  Detail & Related papers  (2026-04-01T16:59:01Z)
• Controllable Exploration in Hybrid-Policy RLVR for Multi-Modal Reasoning [88.42566960813438]
  CalibRL is a hybrid-policy RLVR framework that supports controllable exploration with expert guidance.<n>CalibRL increases policy entropy in a guided manner and clarifies the target distribution.<n>Experiments across eight benchmarks, including both in-domain and out-of-domain settings, demonstrate consistent improvements.
  arXiv  Detail & Related papers  (2026-02-22T07:23:36Z)
• Efficiently Learning Robust Torque-based Locomotion Through Reinforcement with Model-Based Supervision [14.246416873754905]
  We propose a control framework that integrates model-based bipedal locomotion with residual reinforcement learning.<n>Our method demonstrates improved robustness and generalization across a range of randomized conditions.
  arXiv  Detail & Related papers  (2026-01-22T16:56:52Z)
• On-the-Fly VLA Adaptation via Test-Time Reinforcement Learning [61.38273866396522]
  Vision-Language-Action models have emerged as a powerful paradigm for general-purpose robot learning.<n>Current methods remain unsuitable for challenging simulated- or physical-world deployments.<n>We introduce a Test-Time Reinforcement Learning for VLAs framework that enables on-the-fly policy adaptation during inference.
  arXiv  Detail & Related papers  (2026-01-11T01:51:30Z)
• Integrating DeepRL with Robust Low-Level Control in Robotic Manipulators for Non-Repetitive Reaching Tasks [0.24578723416255746]
  In robotics, contemporary strategies are learning-based, characterized by a complex black-box nature and a lack of interpretability. We propose integrating a collision-free trajectory planner based on deep reinforcement learning (DRL) with a novel auto-tuning low-level control strategy.
  arXiv  Detail & Related papers  (2024-02-04T15:54:03Z)
• Training and Evaluation of Deep Policies using Reinforcement Learning and Generative Models [67.78935378952146]
  GenRL is a framework for solving sequential decision-making problems. It exploits the combination of reinforcement learning and latent variable generative models. We experimentally determine the characteristics of generative models that have most influence on the performance of the final policy training.
  arXiv  Detail & Related papers  (2022-04-18T22:02:32Z)
• Robust Deep Reinforcement Learning for Quadcopter Control [0.8687092759073857]
  In this work, we use Robust Markov Decision Processes (RMDP) to train the drone control policy. It opts for pessimistic optimization to handle potential gaps between policy transfer from one environment to another. The trained control policy is tested on the task of quadcopter positional control.
  arXiv  Detail & Related papers  (2021-11-06T16:35:13Z)
• Robust Reinforcement Learning using Adversarial Populations [118.73193330231163]
  Reinforcement Learning (RL) is an effective tool for controller design but can struggle with issues of robustness. We show that using a single adversary does not consistently yield robustness to dynamics variations under standard parametrizations of the adversary. We propose a population-based augmentation to the Robust RL formulation in which we randomly initialize a population of adversaries and sample from the population uniformly during training.
  arXiv  Detail & Related papers  (2020-08-04T20:57:32Z)
• Evolutionary Stochastic Policy Distillation [139.54121001226451]
  We propose a new method called Evolutionary Policy Distillation (ESPD) to solve GCRS tasks. ESPD enables a target policy to learn from a series of its variants through the technique of policy distillation (PD) The experiments based on the MuJoCo control suite show the high learning efficiency of the proposed method.
  arXiv  Detail & Related papers  (2020-04-27T16:19:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.