SARAD: LLM-Based Safety-Aware Hybrid Reinforcement Learning with Collision Prediction for Autonomous Driving

• URL: http://arxiv.org/abs/2605.28583v1
• Date: Wed, 27 May 2026 15:06:34 GMT
• Title: SARAD: LLM-Based Safety-Aware Hybrid Reinforcement Learning with Collision Prediction for Autonomous Driving
• Abstract summary: Traditional Deep Reinforcement Learning (DRL) suffers from unsafe random exploration and slow convergence.<n>Large Language Models (LLMs) demonstrate inherent latency in real-time inference operations.<n>This paper proposes SARAD, a novel safety-aware hybrid framework that synergizes LLMs and DRL for autonomous driving.
• Score: 22.134329848099142
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Ensuring both safety and efficiency in decision-making for autonomous driving systems remains a fundamental challenge. Traditional Deep Reinforcement Learning (DRL) suffers from unsafe random exploration and slow convergence, while Large Language Models (LLMs) demonstrate inherent latency in real-time inference operations. To address these limitations, this paper proposes SARAD, a novel safety-aware hybrid framework that synergizes LLMs and DRL for autonomous driving. SARAD substitutes the random exploration of DRL with Retrieval-Augmented Generation (RAG)-enhanced, LLM-guided decisions sourced from a dynamic expert knowledge repository. An attention discriminator is proposed to integrate the prior knowledge of LLMs into DRL policy optimization. A collision predictor module, fine-tuned with historical collision data, is further designed to improve vehicle safety. Extensive experiments show that SARAD achieves significant performance improvements in the Highway-Env simulator, validating the effectiveness of the proposed model in autonomous driving.

Related papers

• Safe Reinforcement Learning of Autonomous Highway Driving: A Unified Framework for Safety and Efficiency [2.6655248034321724]
  We propose a unified safe reinforcement learning (SRL) framework that integrates safe distance, reward machines (RM), and mixture-of-experts (MoE)<n>Experiments in two-lane traffic show that MoE-RM-SRL substantially improves safety and efficiency over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-06-12T16:32:22Z)
• Large Multimodal Models for Embodied Intelligent Driving: The Next Frontier in Self-Driving? [68.82027978227008]
  This article introduces a novel semantics and policy dual-driven hybrid decision framework to tackle this challenge.<n>The framework merges LMMs for semantic understanding and cognitive representation, and deep reinforcement learning (DRL) for real-time policy optimization.<n>Case study is conducted experimentally to validate the performance superiority of our framework in completing lane-change planning task.
  arXiv  Detail & Related papers  (2026-01-13T11:05:12Z)
• Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Driving [55.13109926181247]
  We introduce ReflectDrive, a learning-based framework that integrates a reflection mechanism for safe trajectory generation via discrete diffusion.<n>Central to our approach is a safety-aware reflection mechanism that performs iterative self-correction without gradient.<n>Our method begins with goal-conditioned trajectory generation to model multi-modal driving behaviors.
  arXiv  Detail & Related papers  (2025-09-24T13:35:15Z)
• SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models [63.71984266104757]
  We propose SafeAuto, a framework that enhances MLLM-based autonomous driving by incorporating both unstructured and structured knowledge.<n>To explicitly integrate safety knowledge, we develop a reasoning component that translates traffic rules into first-order logic.<n>Our Multimodal Retrieval-Augmented Generation model leverages video, control signals, and environmental attributes to learn from past driving experiences.
  arXiv  Detail & Related papers  (2025-02-28T21:53:47Z)
• TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning [61.33599727106222]
  TeLL-Drive is a hybrid framework that integrates a Teacher LLM to guide an attention-based Student DRL policy.<n>A self-attention mechanism then fuses these strategies with the DRL agent's exploration, accelerating policy convergence and boosting robustness.
  arXiv  Detail & Related papers  (2025-02-03T14:22:03Z)
• HighwayLLM: Decision-Making and Navigation in Highway Driving with RL-Informed Language Model [5.4854443795779355]
  This study presents a novel approach, HighwayLLM, which harnesses the reasoning capabilities of large language models (LLMs) to predict the future waypoints for ego-vehicle's navigation. Our approach also utilizes a pre-trained Reinforcement Learning (RL) model to serve as a high-level planner, making decisions on appropriate meta-level actions.
  arXiv  Detail & Related papers  (2024-05-22T11:32:37Z)
• Empowering Autonomous Driving with Large Language Models: A Safety Perspective [82.90376711290808]
  This paper explores the integration of Large Language Models (LLMs) into Autonomous Driving systems. LLMs are intelligent decision-makers in behavioral planning, augmented with a safety verifier shield for contextual safety learning. We present two key studies in a simulated environment: an adaptive LLM-conditioned Model Predictive Control (MPC) and an LLM-enabled interactive behavior planning scheme with a state machine.
  arXiv  Detail & Related papers  (2023-11-28T03:13:09Z)
• Towards Safe Autonomous Driving Policies using a Neuro-Symbolic Deep Reinforcement Learning Approach [5.4854443795779355]
  This paper introduces a novel neuro-symbolic model-free DRL approach, called DRL with Symbolic Logic (DRLSL)<n>It combines the strengths of DRL (learning from experience) and symbolic first-order logic (knowledge-driven reasoning) to enable safe learning in real-time interactions of autonomous driving within real environments.<n>We have implemented the DRLSL framework in a highway driving scenario using the HighD dataset and demonstrated that our method successfully avoids unsafe actions during both the training and testing phases.
  arXiv  Detail & Related papers  (2023-07-03T19:43:21Z)
• Cautious Adaptation For Reinforcement Learning in Safety-Critical Settings [129.80279257258098]
  Reinforcement learning (RL) in real-world safety-critical target settings like urban driving is hazardous. We propose a "safety-critical adaptation" task setting: an agent first trains in non-safety-critical "source" environments. We propose a solution approach, CARL, that builds on the intuition that prior experience in diverse environments equips an agent to estimate risk.
  arXiv  Detail & Related papers  (2020-08-15T01:40:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.