Uncertainty-Aware and Temporally Regulated Expert Advice in Reinforcement Learning for Autonomous Driving

• URL: http://arxiv.org/abs/2605.30576v1
• Date: Thu, 28 May 2026 21:09:44 GMT
• Title: Uncertainty-Aware and Temporally Regulated Expert Advice in Reinforcement Learning for Autonomous Driving
• Abstract summary: Exploration in reinforcement learning for autonomous driving is inherently unsafe.<n>We propose an uncertainty-aware framework that leverages expert advice to guide exploration while avoiding long-term dependence.<n>Our method outperforms the IQN baseline, improving success by 5-7% and reducing failures.
• Score: 9.708935332287265
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Exploration in reinforcement learning for autonomous driving is inherently unsafe: agents must experience novel behaviors to learn, yet exploration can lead to collisions or off-road driving. We propose an uncertainty-aware framework that leverages expert advice to guide exploration while avoiding long-term dependence. Advice is triggered when epistemic or aleatoric uncertainty exceeds adaptive thresholds derived from rolling buffers, ensuring advice evolves with the agent's confidence. A commitment-cooldown strategy with a stochastic early-stop heuristic regulates the duration and frequency of guidance, exposing the agent to coherent maneuvers without exhausting the advice budget. Expert and agent experiences are combined in a shared replay buffer within an off-policy implicit quantile network (IQN) backbone, enabling efficient reuse of expert trajectories. Experiments in CARLA show that our method outperforms the IQN baseline, improving success by 5-7% and reducing failures, demonstrating that risk-sensitive uncertainty coupled with regulated expert integration enables safer and more efficient exploration for sensor-based RL policy learning in unsignalized intersection navigation.

Related papers

• Self-Aware Active Learning Enables Continual Improvement in Autonomous Driving [12.669051115615176]
  Self-aware guided exploration (SAGE) is an active learning framework for post-training adaptation in autonomous driving.<n>Curiosity adaptively calibrates the intervention threshold for fear, allowing the agent to regulate risk in a context-dependent manner.<n>Fear is integrated into policy optimization and evaluation as a safety-oriented constraint to reduce performance regressions.
  arXiv  Detail & Related papers  (2026-08-30T13:01:32Z)
• Trust, Geometry, and Rules: A Credibility-Aware Reinforcement Learning Framework for Safe USV Navigation under Uncertainty [32.34854002457735]
  We propose a framework integrating credibility-aware learning, geometric safety shielding, and continuous rule-aware embedding.<n>We show improved training robustness against perceptual inconsistency and superior collision avoidance and COLREGs compliance over baselines.
  arXiv  Detail & Related papers  (2026-05-26T12:59:43Z)
• Less Approximates More: Harmonizing Performance and Confidence Faithfulness via Hybrid Post-Training for High-Stakes Tasks [52.81286869496811]
  Large language models are increasingly deployed in high-stakes tasks, where confident yet incorrect inferences may cause real-world harm.<n>We propose to jointly optimize unsupervised Reinforcement Learning from Internal Feedback with reasoning-trace-guided Reasoning Distillation.<n>We introduce HyTuning, a hybrid post-training framework that adaptively reweights RD and RLIF via a PRG-style metric.
  arXiv  Detail & Related papers  (2026-04-09T16:50:11Z)
• Expert Knowledge-driven Reinforcement Learning for Autonomous Racing via Trajectory Guidance and Dynamics Constraints [39.69523326789163]
  This paper proposes a guidance trajectory and dynamics constraints Reinforcement Learning (TraD-RL) method for autonomous racing.<n>The proposed method is evaluated in a high-fidelity simulation environment modeled after the Tempelhof Airport Street Circuit.<n> Experimental results demonstrate that TraD-RL effectively improves both lap speed and driving stability of the autonomous racing vehicle.
  arXiv  Detail & Related papers  (2026-03-06T02:51:54Z)
• Safe Reinforcement Learning via Recovery-based Shielding with Gaussian Process Dynamics Models [57.006252510102506]
  Reinforcement learning (RL) is a powerful framework for optimal decision-making and control but often lacks provable guarantees for safety-critical applications.<n>We introduce a novel recovery-based shielding framework that enables safe RL with a provable safety lower bound for unknown and non-linear continuous dynamical systems.
  arXiv  Detail & Related papers  (2026-02-12T22:03:35Z)
• BAPO: Boundary-Aware Policy Optimization for Reliable Agentic Search [72.87861928940929]
  Boundary-Aware Policy Optimization (BAPO) is a novel RL framework designed to cultivate reliable boundary awareness without compromising accuracy.<n>BAPO introduces two key components: (i) a group-based boundary-aware reward that encourages an IDK response only when the reasoning reaches its limit, and (ii) an adaptive reward modulator that strategically suspends this reward during early exploration, preventing the model from exploiting IDK as a shortcut.
  arXiv  Detail & Related papers  (2026-01-16T07:06:58Z)
• Sparse Threats, Focused Defense: Criticality-Aware Robust Reinforcement Learning for Safe Autonomous Driving [11.62520853262219]
  We introduce criticality-aware robust RL (CARRL) for handling sparse, safety-critical risks in autonomous driving.<n>CARRL consists of two interacting components: a risk exposure adversary (REA) and a risk-targeted robust agent (RTRA)<n>We show that our approach reduces the collision rate by at least 22.66% across all cases compared to state-of-the-art baseline methods.
  arXiv  Detail & Related papers  (2026-01-05T05:20:16Z)
• Active Test-time Vision-Language Navigation [60.69722522420299]
  ATENA is a test-time active learning framework that enables a practical human-robot interaction via episodic feedback on uncertain navigation outcomes.<n>In particular, ATENA learns to increase certainty in successful episodes and decrease it in failed ones, improving uncertainty calibration.<n>In addition, we propose a self-active learning strategy that enables an agent to evaluate its navigation outcomes based on confident predictions.
  arXiv  Detail & Related papers  (2025-06-07T02:24:44Z)
• Closing the Closed-Loop Distribution Shift in Safe Imitation Learning [80.05727171757454]
  We treat safe optimization-based control strategies as experts in an imitation learning problem. We train a learned policy that can be cheaply evaluated at run-time and that provably satisfies the same safety guarantees as the expert.
  arXiv  Detail & Related papers  (2021-02-18T05:11:41Z)
• Safe Reinforcement Learning via Curriculum Induction [94.67835258431202]
  In safety-critical applications, autonomous agents may need to learn in an environment where mistakes can be very costly. Existing safe reinforcement learning methods make an agent rely on priors that let it avoid dangerous situations. This paper presents an alternative approach inspired by human teaching, where an agent learns under the supervision of an automatic instructor.
  arXiv  Detail & Related papers  (2020-06-22T10:48:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.