Safe Reinforcement Learning of Autonomous Highway Driving: A Unified Framework for Safety and Efficiency

• URL: http://arxiv.org/abs/2606.14609v1
• Date: Fri, 12 Jun 2026 16:32:22 GMT
• Title: Safe Reinforcement Learning of Autonomous Highway Driving: A Unified Framework for Safety and Efficiency
• Abstract summary: We propose a unified safe reinforcement learning (SRL) framework that integrates safe distance, reward machines (RM), and mixture-of-experts (MoE)<n>Experiments in two-lane traffic show that MoE-RM-SRL substantially improves safety and efficiency over state-of-the-art baselines.
• Score: 2.6655248034321724
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep reinforcement learning (DRL) offers a compelling route to decision-making for advanced autonomous vehicles (AVs), yet its trial-and-error nature makes it difficult to guarantee safety during training and to achieve both safety and efficiency at deployment. We propose a unified safe reinforcement learning (SRL) framework that integrates safe distance (SD), reward machines (RM), and mixture-of-experts (MoE), termed MoE-RM-SRL. For deployment, SD and RM jointly shape a rule-aware reward that encodes highway traffic regulations and stage-wise objectives, enabling safe and reliable behavior without sacrificing efficiency. For training, we introduce a sparsely gated MoE layer comprising up to 11 deep Q-networks (DQNs); an SD-based gating rule activates a minimal set of experts for lane-keeping and lane-changing, mitigating the instability, discontinuities, and impulsive transients commonly induced by switching between heterogeneous controllers (e.g., MPC/rule-based modules and learned policies). We implement the proposed architecture in CARLA and integrate it with a 6-DoF driver-in-the-loop virtual-reality (DiL-VR) platform. Experiments in stochastic two-lane traffic show that MoE-RM-SRL substantially improves safety and efficiency over state-of-the-art baselines, and the framework naturally extends to multi-lane driving as well as on-ramp merging and exiting scenarios.

Related papers

• Reinforcement Learning-Based Control of CAV Platoon Joining Maneuvers in Mixed Traffic [2.455444813386932]
  Connected and automated vehicle (CAV) platooning offers a promising approach to improving road safety and traffic capacity.<n>Reinforcement learning has strong potential for addressing control problems, but its practical deployment raises challenges related to safety and learning efficiency.<n>This paper proposes a generic modeling and simulation framework for investigating CAV platoon joining maneuvers.
  arXiv  Detail & Related papers  (2026-08-27T09:27:11Z)
• Lightweight Safe Reinforcement Learning for End-to-End UAV Navigation [16.850454229093135]
  We propose a safety-constrained perception-control integrated framework for UAV navigation.<n>A lightweight network encodes sparse observations into collision-risk-aware features using asymmetric and depthwise separable convolutions.<n> Experiments with varying obstacle densities and flight speeds demonstrate higher success rates, improved safety, and better efficiency.
  arXiv  Detail & Related papers  (2026-07-02T07:12:24Z)
• Harnessing Textual Refusal Directions for Multimodal Safety [70.16764304975675]
  We introduce Modality-Agnostic Refusal Steering (MARS), a light-weight training-free approach that injects multimodal safety without the need for multimodal safety data.<n>MARS corrects modality misalignment via activation re-centering, adaptively scales steering strength within a geometrically defined trust region, and selects the optimal intervention layer.
  arXiv  Detail & Related papers  (2026-06-30T15:57:50Z)
• An LLM-Explainable DRL Framework for Passenger-Directed Autonomous Driving [2.0646127669654826]
  This work combines deep reinforcement learning (DRL) for adaptive driving control with large language model (LLM)-based explainability modules.<n>DRL agents were trained in simulation using a Dueling Double Deep Q-Network to follow distinct driving requests.<n>LLM modules were introduced to interpret passenger requests, determine when explanations were needed, and generate concise, safety-oriented justifications.
  arXiv  Detail & Related papers  (2026-06-02T15:15:05Z)
• MESA: Improving MoE Safety Alignment via Decentralized Expertise [55.2978630005606]
  MESA (MoE Safety Alignment) is a targeted alignment framework for MoE-based Large Language Models.<n>It strategically decentralizes safety responsibility to maximize coverage while minimizing interference with utility.<n>MESA achieves robust defensive performance against varied harmful benchmarks while preserving helpfulness.
  arXiv  Detail & Related papers  (2026-05-30T09:54:38Z)
• SARAD: LLM-Based Safety-Aware Hybrid Reinforcement Learning with Collision Prediction for Autonomous Driving [22.134329848099142]
  Traditional Deep Reinforcement Learning (DRL) suffers from unsafe random exploration and slow convergence.<n>Large Language Models (LLMs) demonstrate inherent latency in real-time inference operations.<n>This paper proposes SARAD, a novel safety-aware hybrid framework that synergizes LLMs and DRL for autonomous driving.
  arXiv  Detail & Related papers  (2026-05-27T15:06:34Z)
• BarrierSteer: LLM Safety via Learning Barrier Steering [83.12893815611052]
  BarrierSteer is a novel framework that formalizes safety by embedding learned non-linear safety constraints directly into the model's latent representation space.<n>We show that BarrierSteer substantially reduces adversarial success rates, decreases unsafe generations, and outperforms existing methods.
  arXiv  Detail & Related papers  (2026-02-23T18:19:46Z)
• IRL-DAL: Safe and Adaptive Trajectory Planning for Autonomous Driving via Energy-Guided Diffusion Models [0.0]
  This paper proposes a novel inverse reinforcement learning framework using a diffusion-based adaptive lookahead planner (IRL-DAL) for autonomous vehicles.<n>A 96% success rate is reached, and collisions are reduced to 0.05 per 1k steps, marking a new benchmark for safe navigation.
  arXiv  Detail & Related papers  (2026-01-30T18:34:10Z)
• COSMO-RL: Towards Trustworthy LMRMs via Joint Safety and Stability [101.80200069234377]
  We present COSMO-RL, a mixed reinforcement learning framework that trains LMRMs under multimodal, multitask, and multiobjective signals.<n>Our approach aims to let safety and capability grow together in one stable pipeline rather than competing during alignment.
  arXiv  Detail & Related papers  (2025-10-05T13:30:03Z)
• Automating Steering for Safe Multimodal Large Language Models [58.36932318051907]
  We introduce a modular and adaptive inference-time intervention technology, AutoSteer, without requiring any fine-tuning of the underlying model.<n>AutoSteer incorporates three core components: (1) a novel Safety Awareness Score (SAS) that automatically identifies the most safety-relevant distinctions among the model's internal layers; (2) an adaptive safety prober trained to estimate the likelihood of toxic outputs from intermediate representations; and (3) a lightweight Refusal Head that selectively intervenes to modulate generation when safety risks are detected.
  arXiv  Detail & Related papers  (2025-07-17T16:04:55Z)
• SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models [63.71984266104757]
  We propose SafeAuto, a framework that enhances MLLM-based autonomous driving by incorporating both unstructured and structured knowledge.<n>To explicitly integrate safety knowledge, we develop a reasoning component that translates traffic rules into first-order logic.<n>Our Multimodal Retrieval-Augmented Generation model leverages video, control signals, and environmental attributes to learn from past driving experiences.
  arXiv  Detail & Related papers  (2025-02-28T21:53:47Z)
• TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning [61.33599727106222]
  TeLL-Drive is a hybrid framework that integrates a Teacher LLM to guide an attention-based Student DRL policy.<n>A self-attention mechanism then fuses these strategies with the DRL agent's exploration, accelerating policy convergence and boosting robustness.
  arXiv  Detail & Related papers  (2025-02-03T14:22:03Z)
• SafeDrive: Knowledge- and Data-Driven Risk-Sensitive Decision-Making for Autonomous Vehicles with Large Language Models [14.790308656087316]
  SafeDrive is a knowledge- and data-driven risk-sensitive decision-making framework to enhance autonomous driving safety and adaptability.<n>By integrating knowledge-driven insights with adaptive learning mechanisms, the framework ensures robust decision-making under uncertain conditions.
  arXiv  Detail & Related papers  (2024-12-17T16:45:27Z)
• CIMRL: Combining IMitation and Reinforcement Learning for Safe Autonomous Driving [45.05135725542318]
  IMitation and Reinforcement Learning (CIMRL) approach enables training driving policies in simulation through leveraging imitative motion priors and safety constraints. By combining RL and imitation, we demonstrate our method achieves state-of-the-art results in closed loop simulation and real world driving benchmarks.
  arXiv  Detail & Related papers  (2024-06-13T07:31:29Z)
• Safety Correction from Baseline: Towards the Risk-aware Policy in Robotics via Dual-agent Reinforcement Learning [64.11013095004786]
  We propose a dual-agent safe reinforcement learning strategy consisting of a baseline and a safe agent. Such a decoupled framework enables high flexibility, data efficiency and risk-awareness for RL-based control. The proposed method outperforms the state-of-the-art safe RL algorithms on difficult robot locomotion and manipulation tasks.
  arXiv  Detail & Related papers  (2022-12-14T03:11:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.