Automatic Curriculum Learning for Driving Scenarios: Towards Robust and Efficient Reinforcement Learning

• URL: http://arxiv.org/abs/2505.08264v2
• Date: Fri, 11 Jul 2025 09:23:36 GMT
• Title: Automatic Curriculum Learning for Driving Scenarios: Towards Robust and Efficient Reinforcement Learning
• Authors: Ahmed Abouelazm, Tim Weinstein, Tim Joseph, Philip Schörner, J. Marius Zöllner,
• Abstract summary: This paper addresses the challenges of training end-to-end autonomous driving agents using Reinforcement Learning (RL)<n>RL agents are typically trained in a fixed set of scenarios and nominal behavior of surrounding road users in simulations.<n>We propose an automatic curriculum learning framework that dynamically generates driving scenarios with adaptive complexity based on the agent's evolving capabilities.
• Score: 11.602831593017427
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper addresses the challenges of training end-to-end autonomous driving agents using Reinforcement Learning (RL). RL agents are typically trained in a fixed set of scenarios and nominal behavior of surrounding road users in simulations, limiting their generalization and real-life deployment. While domain randomization offers a potential solution by randomly sampling driving scenarios, it frequently results in inefficient training and sub-optimal policies due to the high variance among training scenarios. To address these limitations, we propose an automatic curriculum learning framework that dynamically generates driving scenarios with adaptive complexity based on the agent's evolving capabilities. Unlike manually designed curricula that introduce expert bias and lack scalability, our framework incorporates a ``teacher'' that automatically generates and mutates driving scenarios based on their learning potential -- an agent-centric metric derived from the agent's current policy -- eliminating the need for expert design. The framework enhances training efficiency by excluding scenarios the agent has mastered or finds too challenging. We evaluate our framework in a reinforcement learning setting where the agent learns a driving policy from camera images. Comparative results against baseline methods, including fixed scenario training and domain randomization, demonstrate that our approach leads to enhanced generalization, achieving higher success rates: +9% in low traffic density, +21% in high traffic density, and faster convergence with fewer training steps. Our findings highlight the potential of ACL in improving the robustness and efficiency of RL-based autonomous driving agents.

Related papers

• Diverse and Adaptive Behavior Curriculum for Autonomous Driving: A Student-Teacher Framework with Multi-Agent RL [11.198097218885191]
  This work introduces a novel student-teacher framework for automatic curriculum learning.<n>The teacher, a graph-based multi-agent RL component, adaptively generates traffic behaviors across diverse difficulty levels.<n>Results demonstrate the teacher's ability to generate diverse traffic behaviors.
  arXiv  Detail & Related papers  (2025-07-25T10:35:30Z)
• Scenario-Based Hierarchical Reinforcement Learning for Automated Driving Decision Making [0.27309692684728615]
  Reinforcement Learning approaches can learn comprehensive decision policies directly from experience.<n>Current approaches fail to achieve generalizability for more complex driving tasks and lack learning efficiency.<n>We present Scenario-based Automated Driving Reinforcement Learning (SAD-RL), the first framework that integrates Reinforcement Learning (RL) of hierarchical policy in a scenario-based environment.
  arXiv  Detail & Related papers  (2025-06-28T21:55:59Z)
• Improving Retrospective Language Agents via Joint Policy Gradient Optimization [57.35348425288859]
  RetroAct is a framework that jointly optimize both task-planning and self-reflective evolution capabilities in language agents.<n>We develop a two-stage joint optimization process that integrates imitation learning and reinforcement learning.<n>We conduct extensive experiments across various testing environments, demonstrating RetroAct has substantial improvements in task performance and decision-making processes.
  arXiv  Detail & Related papers  (2025-03-03T12:54:54Z)
• TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning [61.33599727106222]
  TeLL-Drive is a hybrid framework that integrates a Teacher LLM to guide an attention-based Student DRL policy.<n>A self-attention mechanism then fuses these strategies with the DRL agent's exploration, accelerating policy convergence and boosting robustness.
  arXiv  Detail & Related papers  (2025-02-03T14:22:03Z)
• CuRLA: Curriculum Learning Based Deep Reinforcement Learning for Autonomous Driving [1.188383832081829]
  Deep Reinforcement Learning (DRL) agents address this by learning from experience and maximizing rewards.<n>We propose a method that combines DRL with Curriculum Learning for autonomous driving.
  arXiv  Detail & Related papers  (2025-01-09T05:45:03Z)
• From Novice to Expert: LLM Agent Policy Optimization via Step-wise Reinforcement Learning [62.54484062185869]
  We introduce StepAgent, which utilizes step-wise reward to optimize the agent's reinforcement learning process.<n>We propose implicit-reward and inverse reinforcement learning techniques to facilitate agent reflection and policy adjustment.
  arXiv  Detail & Related papers  (2024-11-06T10:35:11Z)
• From Imitation to Exploration: End-to-end Autonomous Driving based on World Model [24.578178308010912]
  RAMBLE is an end-to-end world model-based RL method for driving decision-making.<n>It can handle complex and dynamic traffic scenarios.<n>It achieves state-of-the-art performance in route completion rate on the CARLA Leaderboard 1.0 and completes all 38 scenarios on the CARLA Leaderboard 2.0.
  arXiv  Detail & Related papers  (2024-10-03T06:45:59Z)
• Importance Sampling-Guided Meta-Training for Intelligent Agents in Highly Interactive Environments [43.144056801987595]
  This study introduces a novel training framework that integrates guided meta RL with importance sampling (IS) to optimize training distributions. By estimating a naturalistic distribution from real-world datasets, the framework ensures a balanced focus across common and extreme driving scenarios.
  arXiv  Detail & Related papers  (2024-07-22T17:57:12Z)
• RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashes [57.319845580050924]
  We propose a reinforcement learning framework that combines risk-sensitive control with an adaptive action space curriculum. We show that our algorithm is capable of learning high-speed policies for a real-world off-road driving task.
  arXiv  Detail & Related papers  (2024-05-07T23:32:36Z)
• Rethinking Closed-loop Training for Autonomous Driving [82.61418945804544]
  We present the first empirical study which analyzes the effects of different training benchmark designs on the success of learning agents. We propose trajectory value learning (TRAVL), an RL-based driving agent that performs planning with multistep look-ahead. Our experiments show that TRAVL can learn much faster and produce safer maneuvers compared to all the baselines.
  arXiv  Detail & Related papers  (2023-06-27T17:58:39Z)
• Safety Correction from Baseline: Towards the Risk-aware Policy in Robotics via Dual-agent Reinforcement Learning [64.11013095004786]
  We propose a dual-agent safe reinforcement learning strategy consisting of a baseline and a safe agent. Such a decoupled framework enables high flexibility, data efficiency and risk-awareness for RL-based control. The proposed method outperforms the state-of-the-art safe RL algorithms on difficult robot locomotion and manipulation tasks.
  arXiv  Detail & Related papers  (2022-12-14T03:11:25Z)
• Improving Generalization of Reinforcement Learning with Minimax Distributional Soft Actor-Critic [11.601356612579641]
  This paper introduces the minimax formulation and distributional framework to improve the generalization ability of RL algorithms. We implement our method on the decision-making tasks of autonomous vehicles at intersections and test the trained policy in distinct environments.
  arXiv  Detail & Related papers  (2020-02-13T14:09:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.