Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic

• URL: http://arxiv.org/abs/2111.06318v2
• Date: Fri, 5 Jan 2024 06:37:09 GMT
• Title: Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic
• Authors: Wei Zhou, Dong Chen, Jun Yan, Zhaojian Li, Huilin Yin, Wanchen Ge
• Abstract summary: Lane-changing is a great challenge for autonomous vehicles (AVs) in mixed and dynamic traffic scenarios. In this paper, we formulate the lane-changing decision making of multiple AVs in a mixed-traffic highway environment as a multi-agent reinforcement learning (MARL) problem. Our proposed MARL framework consistently outperforms several state-of-the-art benchmarks in terms of efficiency, safety and driver comfort.
• Score: 16.858651125916133
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autonomous driving has attracted significant research interests in the past two decades as it offers many potential benefits, including releasing drivers from exhausting driving and mitigating traffic congestion, among others. Despite promising progress, lane-changing remains a great challenge for autonomous vehicles (AV), especially in mixed and dynamic traffic scenarios. Recently, reinforcement learning (RL), a powerful data-driven control method, has been widely explored for lane-changing decision makings in AVs with encouraging results demonstrated. However, the majority of those studies are focused on a single-vehicle setting, and lane-changing in the context of multiple AVs coexisting with human-driven vehicles (HDVs) have received scarce attention. In this paper, we formulate the lane-changing decision making of multiple AVs in a mixed-traffic highway environment as a multi-agent reinforcement learning (MARL) problem, where each AV makes lane-changing decisions based on the motions of both neighboring AVs and HDVs. Specifically, a multi-agent advantage actor-critic network (MA2C) is developed with a novel local reward design and a parameter sharing scheme. In particular, a multi-objective reward function is proposed to incorporate fuel efficiency, driving comfort, and safety of autonomous driving. Comprehensive experimental results, conducted under three different traffic densities and various levels of human driver aggressiveness, show that our proposed MARL framework consistently outperforms several state-of-the-art benchmarks in terms of efficiency, safety and driver comfort.

Related papers

• Comprehensive Autonomous Vehicle Optimal Routing With Dynamic Heuristics [0.0]
  The proposed model to improve AV user experience, uses a hybrid AV Network of multiple connected autonomous vehicles. The true optimal solution for this problem is to devise an automated guidance system for vehicles in an AV network. The results are analysed and compared to evaluate the effectiveness of the solution and identify gaps and future enhancements.
  arXiv  Detail & Related papers  (2024-03-17T18:21:56Z)
• Development and Assessment of Autonomous Vehicles in Both Fully Automated and Mixed Traffic Conditions [0.0]
  The paper presents a multi-stage approach, starting with the development of a single AV and progressing to connected AVs. A survey is conducted to validate the driving performance of the AV and will be utilized for a mixed traffic case study. Results show that using deep reinforcement learning, the AV acquired driving behavior that reached human driving performance. The adoption of sharing and caring based V2V communication within AV networks enhances their driving behavior, aids in more effective action planning, and promotes collaborative behavior amongst the AVs.
  arXiv  Detail & Related papers  (2023-12-08T02:40:11Z)
• DRUformer: Enhancing the driving scene Important object detection with driving relationship self-understanding [50.81809690183755]
  Traffic accidents frequently lead to fatal injuries, contributing to over 50 million deaths until 2023. Previous research primarily assessed the importance of individual participants, treating them as independent entities. We introduce Driving scene Relationship self-Understanding transformer (DRUformer) to enhance the important object detection task.
  arXiv  Detail & Related papers  (2023-11-11T07:26:47Z)
• iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement Learning [57.24340061741223]
  We introduce a distributed multi-agent reinforcement learning (MARL) algorithm that can predict trajectories and intents in dense and heterogeneous traffic scenarios. Our approach for intent-aware planning, iPLAN, allows agents to infer nearby drivers' intents solely from their local observations.
  arXiv  Detail & Related papers  (2023-06-09T20:12:02Z)
• LCS-TF: Multi-Agent Deep Reinforcement Learning-Based Intelligent Lane-Change System for Improving Traffic Flow [16.34175752810212]
  Existing intelligent lane-change solutions have primarily focused on optimizing the performance of the ego vehicle. Recent research has seen an increased interest in multi-agent reinforcement learning (MARL)-based approaches. We present a novel hybrid MARL-based intelligent lane-change system for AVs designed to jointly optimize the local performance for the ego vehicle.
  arXiv  Detail & Related papers  (2023-03-16T04:03:17Z)
• Towards Robust On-Ramp Merging via Augmented Multimodal Reinforcement Learning [9.48157144651867]
  We present a novel approach for Robust on-ramp merge of CAVs via Augmented and Multi-modal Reinforcement Learning. Specifically, we formulate the on-ramp merging problem as a Markov decision process (MDP) by taking driving safety, comfort driving behavior, and traffic efficiency into account. To provide reliable merging maneuvers, we simultaneously leverage BSM and surveillance images for multi-modal observation.
  arXiv  Detail & Related papers  (2022-07-21T16:34:57Z)
• Learning energy-efficient driving behaviors by imitating experts [75.12960180185105]
  This paper examines the role of imitation learning in bridging the gap between control strategies and realistic limitations in communication and sensing. We show that imitation learning can succeed in deriving policies that, if adopted by 5% of vehicles, may boost the energy-efficiency of networks with varying traffic conditions by 15% using only local observations.
  arXiv  Detail & Related papers  (2022-06-28T17:08:31Z)
• A Cooperation-Aware Lane Change Method for Autonomous Vehicles [16.937363492078426]
  This paper presents a cooperation-aware lane change method utilizing interactions between vehicles. We first propose an interactive trajectory prediction method to explore possible cooperations between an AV and the others. We then propose a motion planning algorithm based on model predictive control (MPC), which incorporates AV's decision and surrounding vehicles' interactive behaviors into constraints.
  arXiv  Detail & Related papers  (2022-01-26T04:45:45Z)
• Transferable Deep Reinforcement Learning Framework for Autonomous Vehicles with Joint Radar-Data Communications [69.24726496448713]
  We propose an intelligent optimization framework based on the Markov Decision Process (MDP) to help the AV make optimal decisions. We then develop an effective learning algorithm leveraging recent advances of deep reinforcement learning techniques to find the optimal policy for the AV. We show that the proposed transferable deep reinforcement learning framework reduces the obstacle miss detection probability by the AV up to 67% compared to other conventional deep reinforcement learning approaches.
  arXiv  Detail & Related papers  (2021-05-28T08:45:37Z)
• Multi-Modal Fusion Transformer for End-to-End Autonomous Driving [59.60483620730437]
  We propose TransFuser, a novel Multi-Modal Fusion Transformer, to integrate image and LiDAR representations using attention. Our approach achieves state-of-the-art driving performance while reducing collisions by 76% compared to geometry-based fusion.
  arXiv  Detail & Related papers  (2021-04-19T11:48:13Z)
• Divide-and-Conquer for Lane-Aware Diverse Trajectory Prediction [71.97877759413272]
  Trajectory prediction is a safety-critical tool for autonomous vehicles to plan and execute actions. Recent methods have achieved strong performances using Multi-Choice Learning objectives like winner-takes-all (WTA) or best-of-many. Our work addresses two key challenges in trajectory prediction, learning outputs, and better predictions by imposing constraints using driving knowledge.
  arXiv  Detail & Related papers  (2021-04-16T17:58:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.