SLowRL: Safe Low-Rank Adaptation Reinforcement Learning for Locomotion

• URL: http://arxiv.org/abs/2603.17092v1
• Date: Tue, 17 Mar 2026 19:26:48 GMT
• Title: SLowRL: Safe Low-Rank Adaptation Reinforcement Learning for Locomotion
• Abstract summary: We introduce SLowRL, a framework that combines Low-Rank Adaptation (LoRA) with training-time safety enforcement via a recovery policy.<n> Experimental results show that our method achieves a $46.5%$ reduction in fine-tuning time and near-zero safety violations.
• Score: 16.763723249971793
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sim-to-real transfer of locomotion policies often leads to performance degradation due to the inevitable sim-to-real gap. Naively fine-tuning these policies directly on hardware is problematic, as it poses risks of mechanical failure and suffers from high sample inefficiency. In this paper, we address the challenge of safely and efficiently fine-tuning reinforcement learning (RL) policies for dynamic locomotion tasks. Specifically, we focus on fine-tuning policies learned in simulation directly on hardware, while explicitly enforcing safety constraints. In doing so, we introduce SLowRL, a framework that combines Low-Rank Adaptation (LoRA) with training-time safety enforcement via a recovery policy. We evaluate our method both in simulation and on a real Unitree Go2 quadruped robot for jump and trot tasks. Experimental results show that our method achieves a $46.5\%$ reduction in fine-tuning time and near-zero safety violations compared to standard proximal policy optimization (PPO) baselines. Notably, we find that a rank-1 adaptation alone is sufficient to recover pre-trained performance in the real world, while maintaining stable and safe real-world fine-tuning. These results demonstrate the practicality of safe, efficient fine-tuning for dynamic real-world robotic applications.

Related papers

• World Engine: Towards the Era of Post-Training for Autonomous Driving [63.61277741443647]
  We introduce World Engine, a generative framework that reconstructs high-fidelity interactive environments from real-world logs.<n>World Engine substantially reduces failures in rare safety-critical scenarios and yields significantly larger gains than scaling pre-training data alone.<n>When deployed on a production-scale autonomous driving system, the resulting policy reduces simulated collisions.
  arXiv  Detail & Related papers  (2026-06-18T06:28:33Z)
• Human-in-the-loop Online Rejection Sampling for Robotic Manipulation [55.99788088622936]
  Hi-ORS stabilizes value estimation by filtering out negatively rewarded samples during online fine-tuning.<n>Hi-ORS fine-tunes a pi-base policy to master contact-rich manipulation in just 1.5 hours of real-world training.
  arXiv  Detail & Related papers  (2025-10-30T11:53:08Z)
• SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfer [60.19411648245077]
  We propose SPiDR, short for Sim-to-real via Pessimistic Domain Randomization.<n> SPiDR is a scalable algorithm with provable guarantees for safe sim-to-real transfer.<n>We demonstrate that SPiDR effectively ensures safety despite the sim-to-real gap while maintaining strong performance.
  arXiv  Detail & Related papers  (2025-09-23T05:03:00Z)
• SATA: Safe and Adaptive Torque-Based Locomotion Policies Inspired by Animal Learning [10.138425472807368]
  SATA is a bio-inspired framework that mimics key biomechanical principles and adaptive learning mechanisms observed in animal locomotion.<n>Our approach effectively addresses the inherent challenges of learning torque-based policies by significantly improving early-stage exploration.<n>Our experimental results indicate that SATA demonstrates remarkable compliance and safety, even in challenging environments.
  arXiv  Detail & Related papers  (2025-02-18T09:25:37Z)
• RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashes [57.319845580050924]
  We propose a reinforcement learning framework that combines risk-sensitive control with an adaptive action space curriculum. We show that our algorithm is capable of learning high-speed policies for a real-world off-road driving task.
  arXiv  Detail & Related papers  (2024-05-07T23:32:36Z)
• Sim-Anchored Learning for On-the-Fly Adaptation [45.123633153460034]
  Fine-tuning simulation-trained RL agents with real-world data often degrades crucial behaviors due to limited or skewed data distributions.<n>We propose framing live-adaptation as a multi-objective optimization problem, where policy objectives must be satisfied both in simulation and reality.
  arXiv  Detail & Related papers  (2023-01-17T16:16:53Z)
• Safety Correction from Baseline: Towards the Risk-aware Policy in Robotics via Dual-agent Reinforcement Learning [64.11013095004786]
  We propose a dual-agent safe reinforcement learning strategy consisting of a baseline and a safe agent. Such a decoupled framework enables high flexibility, data efficiency and risk-awareness for RL-based control. The proposed method outperforms the state-of-the-art safe RL algorithms on difficult robot locomotion and manipulation tasks.
  arXiv  Detail & Related papers  (2022-12-14T03:11:25Z)
• Safe Reinforcement Learning for Legged Locomotion [18.57681606310315]
  We propose a safe reinforcement learning framework that switches between a safe recovery policy that prevents the robot from entering unsafe states, and a learner policy that is optimized to complete the task. We verify the proposed framework in four locomotion tasks on a simulated and real quadrupedal robot. Our method achieves 48.6% fewer falls and comparable or better rewards than the baseline methods in simulation.
  arXiv  Detail & Related papers  (2022-03-05T01:49:16Z)
• Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning [3.923354711049903]
  We propose a distributional reinforcement learning framework to learn adaptive policies that can tune their level of conservativity at run-time based on the desired comfort and utility. We show that our algorithm learns policies that can still drive reliable when the perception noise is two times higher than the training configuration for automated merging and crossing at occluded intersections.
  arXiv  Detail & Related papers  (2021-07-15T13:36:55Z)
• A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning [23.007323699176467]
  We propose a hierarchical behavior planning framework with a set of low-level safe controllers and a high-level reinforcement learning algorithm (H-CtRL) as a coordinator for the low-level controllers. Safety is guaranteed by the low-level optimization/sampling-based controllers, while the high-level reinforcement learning algorithm makes H-CtRL an adaptive and efficient behavior planner. The proposed H-CtRL is proved to be effective in various realistic simulation scenarios, with satisfying performance in terms of both safety and efficiency.
  arXiv  Detail & Related papers  (2021-01-17T20:45:42Z)
• Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings [129.80279257258098]
  Reinforcement learning (RL) in real-world safety-critical target settings like urban driving is hazardous. We propose a "safety-critical adaptation" task setting: an agent first trains in non-safety-critical "source" environments. We propose a solution approach, CARL, that builds on the intuition that prior experience in diverse environments equips an agent to estimate risk.
  arXiv  Detail & Related papers  (2020-08-15T01:40:59Z)
• Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion [78.46388769788405]
  We introduce guided constrained policy optimization (GCPO), an RL framework based upon our implementation of constrained policy optimization (CPPO) We show that guided constrained RL offers faster convergence close to the desired optimum resulting in an optimal, yet physically feasible, robotic control behavior without the need for precise reward function tuning.
  arXiv  Detail & Related papers  (2020-02-22T10:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.