Mix Q-learning for Lane Changing: A Collaborative Decision-Making Method in Multi-Agent Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2406.09755v1
• Date: Fri, 14 Jun 2024 06:44:19 GMT
• Title: Mix Q-learning for Lane Changing: A Collaborative Decision-Making Method in Multi-Agent Deep Reinforcement Learning
• Authors: Xiaojun Bi, Mingjie He, Yiwen Sun,
• Abstract summary: This paper proposes a method named Mix Q-learning for Lane Changing(MQLC) At the collective level, our method coordinates the individual Q and global Q networks by utilizing global information. At the individual level, we integrated a deep learning-based intent recognition module into our observation and enhanced the decision network.
• Score: 8.455482816036689
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Lane-changing decisions, which are crucial for autonomous vehicle path planning, face practical challenges due to rule-based constraints and limited data. Deep reinforcement learning has become a major research focus due to its advantages in data acquisition and interpretability. However, current models often overlook collaboration, which affects not only impacts overall traffic efficiency but also hinders the vehicle's own normal driving in the long run. To address the aforementioned issue, this paper proposes a method named Mix Q-learning for Lane Changing(MQLC) that integrates a hybrid value Q network, taking into account both collective and individual benefits for the greater good. At the collective level, our method coordinates the individual Q and global Q networks by utilizing global information. This enables agents to effectively balance their individual interests with the collective benefit. At the individual level, we integrated a deep learning-based intent recognition module into our observation and enhanced the decision network. These changes provide agents with richer decision information and more accurate feature extraction for improved lane-changing decisions. This strategy enables the multi-agent system to learn and formulate optimal decision-making strategies effectively. Our MQLC model, through extensive experimental results, impressively outperforms other state-of-the-art multi-agent decision-making methods, achieving significantly safer and faster lane-changing decisions.

Related papers

• SPformer: A Transformer Based DRL Decision Making Method for Connected Automated Vehicles [9.840325772591024]
  We propose a CAV decision-making architecture based on transformer and reinforcement learning algorithms. A learnable policy token is used as the learning medium of the multi-vehicle joint policy. Our model can make good use of all the state information of vehicles in traffic scenario.
  arXiv  Detail & Related papers  (2024-09-23T15:16:35Z)
• Towards Interactive and Learnable Cooperative Driving Automation: a Large Language Model-Driven Decision-Making Framework [79.088116316919]
  Connected Autonomous Vehicles (CAVs) have begun to open road testing around the world, but their safety and efficiency performance in complex scenarios is still not satisfactory. This paper proposes CoDrivingLLM, an interactive and learnable LLM-driven cooperative driving framework.
  arXiv  Detail & Related papers  (2024-09-19T14:36:00Z)
• Research on Autonomous Driving Decision-making Strategies based Deep Reinforcement Learning [8.794428617785869]
  Behavior decision-making subsystem is a key component of the autonomous driving system. In this work, an advanced deep reinforcement learning model is adopted, which can autonomously learn and optimize driving strategies.
  arXiv  Detail & Related papers  (2024-08-06T10:24:54Z)
• Optimising Human-AI Collaboration by Learning Convincing Explanations [62.81395661556852]
  We propose a method for a collaborative system that remains safe by having a human making decisions. Ardent enables efficient and effective decision-making by adapting to individual preferences for explanations.
  arXiv  Detail & Related papers  (2023-11-13T16:00:16Z)
• LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving [87.1164964709168]
  This work employs Large Language Models (LLMs) as a decision-making component for complex autonomous driving scenarios. Extensive experiments demonstrate that our proposed method not only consistently surpasses baseline approaches in single-vehicle tasks, but also helps handle complex driving behaviors even multi-vehicle coordination.
  arXiv  Detail & Related papers  (2023-10-04T17:59:49Z)
• LCS-TF: Multi-Agent Deep Reinforcement Learning-Based Intelligent Lane-Change System for Improving Traffic Flow [16.34175752810212]
  Existing intelligent lane-change solutions have primarily focused on optimizing the performance of the ego vehicle. Recent research has seen an increased interest in multi-agent reinforcement learning (MARL)-based approaches. We present a novel hybrid MARL-based intelligent lane-change system for AVs designed to jointly optimize the local performance for the ego vehicle.
  arXiv  Detail & Related papers  (2023-03-16T04:03:17Z)
• Integrated Decision and Control for High-Level Automated Vehicles by Mixed Policy Gradient and Its Experiment Verification [10.393343763237452]
  This paper presents a self-evolving decision-making system based on the Integrated Decision and Control (IDC) An RL algorithm called constrained mixed policy gradient (CMPG) is proposed to consistently upgrade the driving policy of the IDC. Experiment results show that boosting by data, the system can achieve better driving ability over model-based methods.
  arXiv  Detail & Related papers  (2022-10-19T14:58:41Z)
• Cellular traffic offloading via Opportunistic Networking with Reinforcement Learning [0.5758073912084364]
  We propose an adaptive offloading solution based on the Reinforcement Learning framework. We evaluate and compare the performance of two well-known learning algorithms: Actor-Critic and Q-Learning. Our solution achieves a higher level of offloading with respect to other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-10-01T13:34:12Z)
• End-to-End Intersection Handling using Multi-Agent Deep Reinforcement Learning [63.56464608571663]
  Navigating through intersections is one of the main challenging tasks for an autonomous vehicle. In this work, we focus on the implementation of a system able to navigate through intersections where only traffic signs are provided. We propose a multi-agent system using a continuous, model-free Deep Reinforcement Learning algorithm used to train a neural network for predicting both the acceleration and the steering angle at each time step.
  arXiv  Detail & Related papers  (2021-04-28T07:54:40Z)
• Leveraging Expert Consistency to Improve Algorithmic Decision Support [62.61153549123407]
  We explore the use of historical expert decisions as a rich source of information that can be combined with observed outcomes to narrow the construct gap. We propose an influence function-based methodology to estimate expert consistency indirectly when each case in the data is assessed by a single expert. Our empirical evaluation, using simulations in a clinical setting and real-world data from the child welfare domain, indicates that the proposed approach successfully narrows the construct gap.
  arXiv  Detail & Related papers  (2021-01-24T05:40:29Z)
• Decentralized Reinforcement Learning: Global Decision-Making via Local Economic Transactions [80.49176924360499]
  We establish a framework for directing a society of simple, specialized, self-interested agents to solve sequential decision problems. We derive a class of decentralized reinforcement learning algorithms. We demonstrate the potential advantages of a society's inherent modular structure for more efficient transfer learning.
  arXiv  Detail & Related papers  (2020-07-05T16:41:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.