Expert Knowledge-driven Reinforcement Learning for Autonomous Racing via Trajectory Guidance and Dynamics Constraints

• URL: http://arxiv.org/abs/2603.05842v1
• Date: Fri, 06 Mar 2026 02:51:54 GMT
• Title: Expert Knowledge-driven Reinforcement Learning for Autonomous Racing via Trajectory Guidance and Dynamics Constraints
• Abstract summary: This paper proposes a guidance trajectory and dynamics constraints Reinforcement Learning (TraD-RL) method for autonomous racing.<n>The proposed method is evaluated in a high-fidelity simulation environment modeled after the Tempelhof Airport Street Circuit.<n> Experimental results demonstrate that TraD-RL effectively improves both lap speed and driving stability of the autonomous racing vehicle.
• Score: 39.69523326789163
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning has demonstrated significant potential in the field of autonomous driving. However, it suffers from defects such as training instability and unsafe action outputs when faced with autonomous racing environments characterized by high dynamics and strong nonlinearities. To this end, this paper proposes a trajectory guidance and dynamics constraints Reinforcement Learning (TraD-RL) method for autonomous racing. The key features of this method are as follows: 1) leveraging the prior expert racing line to construct an augmented state representation and facilitate reward shaping, thereby integrating domain knowledge to stabilize early-stage policy learning; 2) embedding explicit vehicle dynamic priors into a safe operating envelope formulated via control barrier functions to enable safety-constrained learning; and 3) adopting a multi-stage curriculum learning strategy that shifts from expert-guided learning to autonomous exploration, allowing the learned policy to surpass expert-level performance. The proposed method is evaluated in a high-fidelity simulation environment modeled after the Tempelhof Airport Street Circuit. Experimental results demonstrate that TraD-RL effectively improves both lap speed and driving stability of the autonomous racing vehicle, achieving a synergistic optimization of racing performance and safety.

Related papers

• Knowledge-Data-Dual-Driven Reinforcement Learning for Autonomous Vehicle Control in Mixed Traffic [7.833666254318349]
  In mixed traffic, decision-making for autonomous vehicles (AVs) confronts three interrelated challenges.<n>First, physics-based priors fail to capture latent interactive vehicle intentions and diverse driver behaviors.<n>Second, abrupt maneuvers by surrounding vehicles cause non-stationarity, leaving long-tail safety events under-explored.<n>Third, hybrid action spaces destabilize unified RL training due to the different temporal scales of continuous car-following and discrete lane-changing maneuvers.
  arXiv  Detail & Related papers  (2026-08-14T02:07:13Z)
• From Attacks to Curricula: Learnability-Guided Adversarial Training for Safe Autonomous Driving [56.30087557121323]
  AlignADV is a learnability-guided closed-loop adversarial training framework.<n>It converts adversarial scenarios into resolvable and capability-aligned curricula.<n> Experiments show it reduces training steps by up to 40.6 percent.
  arXiv  Detail & Related papers  (2026-06-12T02:13:55Z)
• Behavior-Constrained Reinforcement Learning with Receding-Horizon Credit Assignment for High-Performance Control [13.52467502450336]
  Reinforcement learning can discover high-performing strategies but often departs from desirable human behavior.<n>We propose a behavior-constrained reinforcement learning framework that improves beyond demonstrations.<n>We evaluate the approach in high-fidelity race car simulation using data from professional drivers.
  arXiv  Detail & Related papers  (2026-04-03T13:13:47Z)
• Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Driving [55.13109926181247]
  We introduce ReflectDrive, a learning-based framework that integrates a reflection mechanism for safe trajectory generation via discrete diffusion.<n>Central to our approach is a safety-aware reflection mechanism that performs iterative self-correction without gradient.<n>Our method begins with goal-conditioned trajectory generation to model multi-modal driving behaviors.
  arXiv  Detail & Related papers  (2025-09-24T13:35:15Z)
• TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning [61.33599727106222]
  TeLL-Drive is a hybrid framework that integrates a Teacher LLM to guide an attention-based Student DRL policy.<n>A self-attention mechanism then fuses these strategies with the DRL agent's exploration, accelerating policy convergence and boosting robustness.
  arXiv  Detail & Related papers  (2025-02-03T14:22:03Z)
• RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashes [57.319845580050924]
  We propose a reinforcement learning framework that combines risk-sensitive control with an adaptive action space curriculum. We show that our algorithm is capable of learning high-speed policies for a real-world off-road driving task.
  arXiv  Detail & Related papers  (2024-05-07T23:32:36Z)
• Long and Short-Term Constraints Driven Safe Reinforcement Learning for Autonomous Driving [11.072917563013428]
  Reinforcement learning (RL) has been widely used in decision-making and control tasks, but the risk is very high for the agent in the training process. In this paper, we propose a novel algorithm based on the long and short-term constraints (LSTC) for safe RL. The proposed method achieves higher safety in continuous state and action tasks, and exhibits higher exploration performance in long-distance decision-making tasks.
  arXiv  Detail & Related papers  (2024-03-27T02:41:52Z)
• Deep Reinforcement Learning for Autonomous Vehicle Intersection Navigation [0.24578723416255746]
  Reinforcement learning algorithms have emerged as a promising approach to address these challenges. Here, we address the problem of efficiently and safely navigating T-intersections using a lower-cost, single-agent approach. Our results reveal that the proposed approach enables the AV to effectively navigate T-intersections, outperforming previous methods in terms of travel delays, collision minimization, and overall cost.
  arXiv  Detail & Related papers  (2023-09-30T10:54:02Z)
• Optimizing Trajectories for Highway Driving with Offline Reinforcement Learning [11.970409518725491]
  We propose a Reinforcement Learning-based approach to autonomous driving. We compare the performance of our agent against four other highway driving agents. We demonstrate that our offline trained agent, with randomly collected data, learns to drive smoothly, achieving as close as possible to the desired velocity, while outperforming the other agents.
  arXiv  Detail & Related papers  (2022-03-21T13:13:08Z)
• Differentiable Control Barrier Functions for Vision-based End-to-End Autonomous Driving [100.57791628642624]
  We introduce a safety guaranteed learning framework for vision-based end-to-end autonomous driving. We design a learning system equipped with differentiable control barrier functions (dCBFs) that is trained end-to-end by gradient descent.
  arXiv  Detail & Related papers  (2022-03-04T16:14:33Z)
• UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planning [1.1339580074756188]
  Offline reinforcement learning (RL) provides a framework for learning decision-making from offline data. Self-driving vehicles (SDV) learn a policy, which potentially even outperforms the behavior in the sub-optimal data set. This motivates the use of model-based offline RL approaches, which leverage planning.
  arXiv  Detail & Related papers  (2021-11-22T10:37:52Z)
• Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learning [39.757652701917166]
  This work proposes a new learning-based method to tackle the autonomous overtaking problem. We evaluate our approach using Gran Turismo Sport -- a world-leading car racing simulator.
  arXiv  Detail & Related papers  (2021-03-26T18:06:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.