CycleRL: Sim-to-Real Deep Reinforcement Learning for Robust Autonomous Bicycle Control

• URL: http://arxiv.org/abs/2603.15013v1
• Date: Mon, 16 Mar 2026 09:17:51 GMT
• Title: CycleRL: Sim-to-Real Deep Reinforcement Learning for Robust Autonomous Bicycle Control
• Abstract summary: CycleRL is the first sim-to-real deep reinforcement learning framework for robust autonomous bicycle control.<n>In simulation, CycleRL achieves considerable performance, including a 99.90% balance success rate, a low steering tracking error of 1.15, and a velocity tracking error of 0.18 m/s.
• Score: 10.350603824555408
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autonomous bicycles offer a promising agile solution for urban mobility and last-mile logistics, however, conventional control strategies often struggle with their underactuated nonlinear dynamics, suffering from sensitivity to model mismatches and limited adaptability to real-world uncertainties. To address this, this paper presents CycleRL, the first sim-to-real deep reinforcement learning framework designed for robust autonomous bicycle control. Our approach trains an end-to-end neural control policy within the high-fidelity NVIDIA Isaac Sim environment, leveraging Proximal Policy Optimization (PPO) to circumvent the need for an explicit dynamics model. The framework features a composite reward function tailored for concurrent balance maintenance, velocity tracking, and steering control. Crucially, systematic domain randomization is employed to bridge the simulation-to-reality gap and facilitate direct transfer. In simulation, CycleRL achieves considerable performance, including a 99.90% balance success rate, a low steering tracking error of 1.15°, and a velocity tracking error of 0.18 m/s. These quantitative results, coupled with successful hardware transfer, validate DRL as an effective paradigm for autonomous bicycle control, offering superior adaptability over traditional methods. Video demonstrations are available at https://anony6f05.github.io/CycleRL/.

Related papers

• A Neural Network Based Teleoperation for Remote Controlled Vehicles [98.38733381483934]
  Direct teleoperation of vehicles faces critical technical bottlenecks.<n> communication latency and the operator's inability to physically perceive unmodeled environmental disturbances.<n>We propose a tailored unilateral teleoperation framework.
  arXiv  Detail & Related papers  (2026-08-11T01:51:08Z)
• ECoSim: Data Efficient Fine-Tuning for Controllable Traffic Simulation [48.30643653308979]
  Controllable traffic simulation is critical for testing autonomous driving systems.<n>Existing approaches often require retraining large generative models with extensive annotated data.<n>We introduce a lightweight control adaptation framework that enables multi-modal controllability.
  arXiv  Detail & Related papers  (2026-07-01T07:35:29Z)
• Physics-Informed Reinforcement Learning of Spatial Density Velocity Potentials for Map-Free Racing [73.88859384645264]
  Out-Of-Distribution (OOD) generalization to various racetracks utilizes Machine Learning (ML) to encode the mathematical relation between sensor data and vehicle actuation for end-to-end control.<n>This paper presents a DRL method that parameterizes nonlinear vehicle dynamics from the spectral distribution of depth measurements with a non-geometric, physics-informed reward, to infer vehicle time-optimal and overtaking racing controls with an Artificial Neural Network (ANN)<n>The policy outperforms human demonstrations by 12% in OOD tracks on proportionally scaled hardware, by maximizing the friction circle with tire dynamics that resemble an empirical Pacejka tire model.
  arXiv  Detail & Related papers  (2026-04-10T17:12:07Z)
• Sim-to-reality adaptation for Deep Reinforcement Learning applied to an underwater docking application [0.0]
  Deep Reinforcement Learning (DRL) offers a robust alternative to traditional control methods for autonomous underwater docking.<n>This paper presents a systematic approach for autonomous docking using the Girona Autonomous Underwater Vehicle (AUV) by leveraging a high-fidelity digital twin environment.
  arXiv  Detail & Related papers  (2026-03-12T15:01:25Z)
• Efficiently Learning Robust Torque-based Locomotion Through Reinforcement with Model-Based Supervision [14.246416873754905]
  We propose a control framework that integrates model-based bipedal locomotion with residual reinforcement learning.<n>Our method demonstrates improved robustness and generalization across a range of randomized conditions.
  arXiv  Detail & Related papers  (2026-01-22T16:56:52Z)
• SPACeR: Self-Play Anchoring with Centralized Reference Models [50.55045557371374]
  Sim agent policies are realistic, human-like, fast, and scalable in multi-agent settings.<n>Recent progress in imitation learning with large diffusion-based or tokenized models has shown that behaviors can be captured directly from human driving data.<n>We propose SPACeR, a framework that leverages a pretrained tokenized autoregressive motion model as a central reference policy.
  arXiv  Detail & Related papers  (2025-10-20T19:53:02Z)
• Advancing Multi-agent Traffic Simulation via R1-Style Reinforcement Fine-Tuning [35.83999932977034]
  We propose a novel R1-style reinforcement fine-tuning paradigm tailored for next-token prediction models to better align agent behavior with human preferences and evaluation metrics.<n>Our approach introduces a metric-oriented policy optimization algorithm to improve distribution alignment and an iterative "SFT-RFT-SFT" training strategy that alternates between Supervised Fine-Tuning (SFT) and Reinforcement Fine-Tuning (RFT)<n>The results on the Open Sim Agents Challenge showcase that SMART-R1 achieves state-of-the-art performance with an overall realism meta score of 0.7858, ranking first on
  arXiv  Detail & Related papers  (2025-09-28T17:36:13Z)
• RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulation [13.319344167881383]
  We introduce a dual-stage AV-centric simulation framework that conducts imitation learning pre-training in a data-driven simulator.<n>We then learn fine-tuning in a physics-based simulator to enhance style-level controllability.<n>In the fine-tuning stage, we propose RIFT, a novel group-relative RL fine-tuning strategy.
  arXiv  Detail & Related papers  (2025-05-06T09:12:37Z)
• Offline Robotic World Model: Learning Robotic Policies without a Physics Simulator [50.191655141020505]
  Reinforcement Learning (RL) has demonstrated impressive capabilities in robotic control but remains challenging due to high sample complexity, safety concerns, and the sim-to-real gap.<n>We introduce Offline Robotic World Model (RWM-O), a model-based approach that explicitly estimates uncertainty to improve policy learning without reliance on a physics simulator.
  arXiv  Detail & Related papers  (2025-04-23T12:58:15Z)
• Physics Enhanced Residual Policy Learning (PERPL) for safety cruising in mixed traffic platooning under actuator and communication delay [8.172286651098027]
  Linear control models have gained extensive application in vehicle control due to their simplicity, ease of use, and support for stability analysis. Reinforcement learning (RL) models, on the other hand, offer adaptability but suffer from a lack of interpretability and generalization capabilities. This paper aims to develop a family of RL-based controllers enhanced by physics-informed policies.
  arXiv  Detail & Related papers  (2024-09-23T23:02:34Z)
• Autonomous Vehicle Controllers From End-to-End Differentiable Simulation [57.278726604424556]
  We propose a differentiable simulator and design an analytic policy gradients (APG) approach to training AV controllers.<n>Our proposed framework brings the differentiable simulator into an end-to-end training loop, where gradients of environment dynamics serve as a useful prior to help the agent learn a more grounded policy.<n>We find significant improvements in performance and robustness to noise in the dynamics, as well as overall more intuitive human-like handling.
  arXiv  Detail & Related papers  (2024-09-12T11:50:06Z)
• Reinforcement Learning with Human Feedback for Realistic Traffic Simulation [53.85002640149283]
  Key element of effective simulation is the incorporation of realistic traffic models that align with human knowledge. This study identifies two main challenges: capturing the nuances of human preferences on realism and the unification of diverse traffic simulation models.
  arXiv  Detail & Related papers  (2023-09-01T19:29:53Z)
• Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots [121.42930679076574]
  We present a model-free reinforcement learning framework for training robust locomotion policies in simulation. domain randomization is used to encourage the policies to learn behaviors that are robust across variations in system dynamics. We demonstrate this on versatile walking behaviors such as tracking a target walking velocity, walking height, and turning yaw.
  arXiv  Detail & Related papers  (2021-03-26T07:14:01Z)
• Information Theoretic Model Predictive Q-Learning [64.74041985237105]
  We present a novel theoretical connection between information theoretic MPC and entropy regularized RL. We develop a Q-learning algorithm that can leverage biased models.
  arXiv  Detail & Related papers  (2019-12-31T00:29:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.