LCS-TF: Multi-Agent Deep Reinforcement Learning-Based Intelligent Lane-Change System for Improving Traffic Flow

• URL: http://arxiv.org/abs/2303.09070v1
• Date: Thu, 16 Mar 2023 04:03:17 GMT
• Title: LCS-TF: Multi-Agent Deep Reinforcement Learning-Based Intelligent Lane-Change System for Improving Traffic Flow
• Abstract summary: Existing intelligent lane-change solutions have primarily focused on optimizing the performance of the ego vehicle. Recent research has seen an increased interest in multi-agent reinforcement learning (MARL)-based approaches. We present a novel hybrid MARL-based intelligent lane-change system for AVs designed to jointly optimize the local performance for the ego vehicle.
• Score: 16.34175752810212
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Discretionary lane-change is one of the critical challenges for autonomous vehicle (AV) design due to its significant impact on traffic efficiency. Existing intelligent lane-change solutions have primarily focused on optimizing the performance of the ego-vehicle, thereby suffering from limited generalization performance. Recent research has seen an increased interest in multi-agent reinforcement learning (MARL)-based approaches to address the limitation of the ego vehicle-based solutions through close coordination of multiple agents. Although MARL-based approaches have shown promising results, the potential impact of lane-change decisions on the overall traffic flow of a road segment has not been fully considered. In this paper, we present a novel hybrid MARL-based intelligent lane-change system for AVs designed to jointly optimize the local performance for the ego vehicle, along with the global performance focused on the overall traffic flow of a given road segment. With a careful review of the relevant transportation literature, a novel state space is designed to integrate both the critical local traffic information pertaining to the surrounding vehicles of the ego vehicle, as well as the global traffic information obtained from a road-side unit (RSU) responsible for managing a road segment. We create a reward function to ensure that the agents make effective lane-change decisions by considering the performance of the ego vehicle and the overall improvement of traffic flow. A multi-agent deep Q-network (DQN) algorithm is designed to determine the optimal policy for each agent to effectively cooperate in performing lane-change maneuvers. LCS-TF's performance was evaluated through extensive simulations in comparison with state-of-the-art MARL models. In all aspects of traffic efficiency, driving safety, and driver comfort, the results indicate that LCS-TF exhibits superior performance.

Related papers

• CoordLight: Learning Decentralized Coordination for Network-Wide Traffic Signal Control [15.468107943726617]
  CoordLight is a framework designed to improve intra-neighborhood traffic by enhancing decision-making at individual junctions (agents)<n>We propose an advanced MARL algorithm, named Neighbor-aware Policy Optimization (NAPO), which integrates an attention mechanism that discerns the state and action dependencies among adjacent agents.<n>We empirically show that CoordLight consistently exhibits superior performance across diverse traffic networks with varying traffic flows.
  arXiv  Detail & Related papers  (2026-03-25T14:46:31Z)
• Scalable Ride-Sourcing Vehicle Rebalancing with Service Accessibility Guarantee: A Constrained Mean-Field Reinforcement Learning Approach [42.070187224580344]
  We introduce continuous-state mean-field control (MFC) and mean-field reinforcement learning (MFRL) models that employ continuous vehicle repositioning actions.<n>MFC and MFRL offer scalable solutions by modeling each vehicle's behavior through interaction with the vehicle distribution, rather than with individual vehicles.<n>Our approach scales to tens of thousands of vehicles, with training times comparable to the decision time of a single linear programming rebalancing.
  arXiv  Detail & Related papers  (2025-03-31T15:00:11Z)
• SPformer: A Transformer Based DRL Decision Making Method for Connected Automated Vehicles [9.840325772591024]
  We propose a CAV decision-making architecture based on transformer and reinforcement learning algorithms. A learnable policy token is used as the learning medium of the multi-vehicle joint policy. Our model can make good use of all the state information of vehicles in traffic scenario.
  arXiv  Detail & Related papers  (2024-09-23T15:16:35Z)
• AgentsCoMerge: Large Language Model Empowered Collaborative Decision Making for Ramp Merging [46.69777653051523]
  Ramp merging is one of the bottlenecks in traffic systems, which commonly cause traffic congestion, accidents, and severe carbon emissions.<n>We propose a novel collaborative decision-making framework, named AgentsCoMerge, to leverage large language models (LLMs)
  arXiv  Detail & Related papers  (2024-08-07T08:34:48Z)
• Mix Q-learning for Lane Changing: A Collaborative Decision-Making Method in Multi-Agent Deep Reinforcement Learning [8.455482816036689]
  This paper proposes a method named Mix Q-learning for Lane Changing(MQLC) At the collective level, our method coordinates the individual Q and global Q networks by utilizing global information. At the individual level, we integrated a deep learning-based intent recognition module into our observation and enhanced the decision network.
  arXiv  Detail & Related papers  (2024-06-14T06:44:19Z)
• A Holistic Framework Towards Vision-based Traffic Signal Control with Microscopic Simulation [53.39174966020085]
  Traffic signal control (TSC) is crucial for reducing traffic congestion that leads to smoother traffic flow, reduced idling time, and mitigated CO2 emissions. In this study, we explore the computer vision approach for TSC that modulates on-road traffic flows through visual observation. We introduce a holistic traffic simulation framework called TrafficDojo towards vision-based TSC and its benchmarking.
  arXiv  Detail & Related papers  (2024-03-11T16:42:29Z)
• DenseLight: Efficient Control for Large-scale Traffic Signals with Dense Feedback [109.84667902348498]
  Traffic Signal Control (TSC) aims to reduce the average travel time of vehicles in a road network. Most prior TSC methods leverage deep reinforcement learning to search for a control policy. We propose DenseLight, a novel RL-based TSC method that employs an unbiased reward function to provide dense feedback on policy effectiveness.
  arXiv  Detail & Related papers  (2023-06-13T05:58:57Z)
• Studying the Impact of Semi-Cooperative Drivers on Overall Highway Flow [76.38515853201116]
  Semi-cooperative behaviors are intrinsic properties of human drivers and should be considered for autonomous driving. New autonomous planners can consider the social value orientation (SVO) of human drivers to generate socially-compliant trajectories. We present study of implicit semi-cooperative driving where agents deploy a game-theoretic version of iterative best response.
  arXiv  Detail & Related papers  (2023-04-23T16:01:36Z)
• SocialLight: Distributed Cooperation Learning towards Network-Wide Traffic Signal Control [7.387226437589183]
  SocialLight is a new multi-agent reinforcement learning method for traffic signal control. It learns cooperative traffic control policies by estimating the individual marginal contribution of agents on their local neighborhood. We benchmark our trained network against state-of-the-art traffic signal control methods on standard benchmarks in two traffic simulators.
  arXiv  Detail & Related papers  (2023-04-20T12:41:25Z)
• Decision Making for Autonomous Driving in Interactive Merge Scenarios via Learning-based Prediction [39.48631437946568]
  This paper focuses on the complex task of merging into moving traffic where uncertainty emanates from the behavior of other drivers. We frame the problem as a partially observable Markov decision process (POMDP) and solve it online with Monte Carlo tree search. The solution to the POMDP is a policy that performs high-level driving maneuvers, such as giving way to an approaching car, keeping a safe distance from the vehicle in front or merging into traffic.
  arXiv  Detail & Related papers  (2023-03-29T16:12:45Z)
• RLPG: Reinforcement Learning Approach for Dynamic Intra-Platoon Gap Adaptation for Highway On-Ramp Merging [14.540226579203207]
  A platoon refers to a group of vehicles traveling together in very close proximity. Recent research has revealed a detrimental effect of the extremely small intra-platoon gap on traffic flow for highway on-ramp merging. We present a novel reinforcement learning framework that adaptively adjusts the intra-platoon gap of an individual platoon member to maximize traffic flow.
  arXiv  Detail & Related papers  (2022-12-07T07:33:54Z)
• AI-aided Traffic Control Scheme for M2M Communications in the Internet of Vehicles [61.21359293642559]
  The dynamics of traffic and the heterogeneous requirements of different IoV applications are not considered in most existing studies. We consider a hybrid traffic control scheme and use proximal policy optimization (PPO) method to tackle it.
  arXiv  Detail & Related papers  (2022-03-05T10:54:05Z)
• Decentralized Cooperative Lane Changing at Freeway Weaving Areas Using Multi-Agent Deep Reinforcement Learning [1.6752182911522522]
  Frequent lane changes during congestion at freeway bottlenecks such as merge and weaving areas further reduce roadway capacity. The emergence of deep reinforcement learning (RL) and connected and automated vehicle technology provides a possible solution to improve mobility and energy efficiency at freeway bottlenecks through cooperative lane changing. In this study, a decentralized cooperative lane-changing controller was developed using a multi-agent deep RL paradigm. The results of this study show that cooperative lane changing enabled by multi-agent deep RL yields superior performance to human drivers in term of traffic throughput, vehicle speed, number of stops per vehicle, vehicle fuel efficiency, and emissions.
  arXiv  Detail & Related papers  (2021-10-05T18:29:13Z)
• End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learning [63.56464608571663]
  Navigating through intersections is one of the main challenging tasks for an autonomous vehicle. In this work, we focus on the implementation of a system able to navigate through intersections where only traffic signs are provided. We propose a multi-agent system using a continuous, model-free Deep Reinforcement Learning algorithm used to train a neural network for predicting both the acceleration and the steering angle at each time step.
  arXiv  Detail & Related papers  (2021-04-28T07:54:40Z)
• MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Control [54.162449208797334]
  Traffic signal control aims to coordinate traffic signals across intersections to improve the traffic efficiency of a district or a city. Deep reinforcement learning (RL) has been applied to traffic signal control recently and demonstrated promising performance where each traffic signal is regarded as an agent. We propose a novel Meta Variationally Intrinsic Motivated (MetaVIM) RL method to learn the decentralized policy for each intersection that considers neighbor information in a latent way.
  arXiv  Detail & Related papers  (2021-01-04T03:06:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.