Continual-RL for Generalization in Autonomous Racing on the RoboRacer Platform

• URL: http://arxiv.org/abs/2607.24320v1
• Date: Mon, 27 Jul 2026 12:01:29 GMT
• Title: Continual-RL for Generalization in Autonomous Racing on the RoboRacer Platform
• Abstract summary: A key challenge in modern robotics is to adapt to changing environments.<n>In particular, learning to drive on a new track-floor combination with the least amount of new experience naturally frames a continual learning problem.<n>This work proposes a continual RL framework based on Continual Backpropagation that is able to train a generalistic policy on a set of tracks and then fine- tune it within 15 minutes to outperform classical controllers.
• Score: 6.737714968545043
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A key challenge in modern robotics is to adapt to changing environments, a challenge that is exacerbated when simulations cannot encompass every possible real-world configuration, and therefore Reinforcement Learning (RL) in the physical world becomes necessary. Continual Reinforcement Learning provides the tools to address this challenge; however, both the frameworks and the methods remain underexplored. Autonomous Racing and in particular the RoboRacer competition provide a testing ground for such methods, as learning to drive on a new track-floor combination with the least amount of new experience naturally frames a continual learning problem. This work tries to address this gap by proposing a continual RL framework based on Continual Backpropagation that is able, with only real-world data, to train a generalistic policy on a set of tracks and then fine- tune it within 15 minutes to outperform classical controllers. Furthermore, a comparison method based on offline RL is proposed, and a simulation analysis of the plasticity properties of the methods is conducted.

Related papers

• Incremental Residual Reinforcement Learning Toward Real-World Learning for Social Navigation [2.1204495827342438]
  Social navigation has emerged as a critical task, driving research into deep reinforcement learning (RL) approaches.<n>Real-world RL, in which agents learn while operating directly in physical environments, presents a promising solution to this issue.<n>In this study, we propose incremental residual RL (IRRL), which enhances learning efficiency by training only on the residuals relative to a base policy.
  arXiv  Detail & Related papers  (2026-04-09T08:08:37Z)
• Reset-Free Reinforcement Learning for Real-World Agile Driving: An Empirical Study [3.038642416291856]
  This paper presents an empirical study of reset-free reinforcement learning (RL) for real-world agile driving.<n>We employ Model Predictive Path Integral control (MPPI) as both the reset policy and the base policy for residual learning.<n>Our results reveal a clear gap between simulation and real-world: SAC with residual learning achieves the highest returns in simulation, yet only TD-MPC2 consistently outperforms the MPPI baseline on the physical platform.
  arXiv  Detail & Related papers  (2026-04-09T00:28:54Z)
• Offline Robotic World Model: Learning Robotic Policies without a Physics Simulator [50.191655141020505]
  Reinforcement Learning (RL) has demonstrated impressive capabilities in robotic control but remains challenging due to high sample complexity, safety concerns, and the sim-to-real gap.<n>We introduce Offline Robotic World Model (RWM-O), a model-based approach that explicitly estimates uncertainty to improve policy learning without reliance on a physics simulator.
  arXiv  Detail & Related papers  (2025-04-23T12:58:15Z)
• A General Infrastructure and Workflow for Quadrotor Deep Reinforcement Learning and Reality Deployment [48.90852123901697]
  We propose a platform that enables seamless transfer of end-to-end deep reinforcement learning (DRL) policies to quadrotors.<n>Our platform provides rich types of environments including hovering, dynamic obstacle avoidance, trajectory tracking, balloon hitting, and planning in unknown environments.
  arXiv  Detail & Related papers  (2025-04-21T14:25:23Z)
• TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning [61.33599727106222]
  TeLL-Drive is a hybrid framework that integrates a Teacher LLM to guide an attention-based Student DRL policy.<n>A self-attention mechanism then fuses these strategies with the DRL agent's exploration, accelerating policy convergence and boosting robustness.
  arXiv  Detail & Related papers  (2025-02-03T14:22:03Z)
• Continual Task Learning through Adaptive Policy Self-Composition [54.95680427960524]
  CompoFormer is a structure-based continual transformer model that adaptively composes previous policies via a meta-policy network. Our experiments reveal that CompoFormer outperforms conventional continual learning (CL) methods, particularly in longer task sequences.
  arXiv  Detail & Related papers  (2024-11-18T08:20:21Z)
• Sim-to-Real Transfer of Deep Reinforcement Learning Agents for Online Coverage Path Planning [22.077058792635313]
  Coverage path planning is the problem of finding a path that covers the entire free space of a confined area.<n>We investigate the suitability of continuous-space reinforcement learning for this challenging problem.<n>We show that our approach surpasses the performance of both previous RL-based approaches and highly specialized methods.
  arXiv  Detail & Related papers  (2024-06-07T13:24:19Z)
• Aquatic Navigation: A Challenging Benchmark for Deep Reinforcement Learning [53.3760591018817]
  We propose a new benchmarking environment for aquatic navigation using recent advances in the integration between game engines and Deep Reinforcement Learning. Specifically, we focus on PPO, one of the most widely accepted algorithms, and we propose advanced training techniques. Our empirical evaluation shows that a well-designed combination of these ingredients can achieve promising results.
  arXiv  Detail & Related papers  (2024-05-30T23:20:23Z)
• End-to-end Lidar-Driven Reinforcement Learning for Autonomous Racing [0.0]
  Reinforcement Learning (RL) has emerged as a transformative approach in the domains of automation and robotics. This study develops and trains an RL agent to navigate a racing environment solely using feedforward raw lidar and velocity data. The agent's performance is then experimentally evaluated in a real-world racing scenario.
  arXiv  Detail & Related papers  (2023-09-01T07:03:05Z)
• Cloud-Edge Training Architecture for Sim-to-Real Deep Reinforcement Learning [0.8399688944263843]
  Deep reinforcement learning (DRL) is a promising approach to solve complex control tasks by learning policies through interactions with the environment. Sim-to-real approaches leverage simulations to pretrain DRL policies and then deploy them in the real world. This work proposes a distributed cloud-edge architecture to train DRL agents in the real world in real-time.
  arXiv  Detail & Related papers  (2022-03-04T10:27:01Z)
• Autonomous Reinforcement Learning: Formalism and Benchmarking [106.25788536376007]
  Real-world embodied learning, such as that performed by humans and animals, is situated in a continual, non-episodic world. Common benchmark tasks in RL are episodic, with the environment resetting between trials to provide the agent with multiple attempts. This discrepancy presents a major challenge when attempting to take RL algorithms developed for episodic simulated environments and run them on real-world platforms.
  arXiv  Detail & Related papers  (2021-12-17T16:28:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.