Graph Convolution-Based Deep Reinforcement Learning for Multi-Agent
Decision-Making in Mixed Traffic Environments
- URL: http://arxiv.org/abs/2201.12776v1
- Date: Sun, 30 Jan 2022 10:09:43 GMT
- Title: Graph Convolution-Based Deep Reinforcement Learning for Multi-Agent
Decision-Making in Mixed Traffic Environments
- Authors: Qi Liu, Zirui Li, Xueyuan Li, Jingda Wu, Shihua Yuan
- Abstract summary: This research proposes a framework to enable different Graph Reinforcement Learning (GRL) methods for decision-making.
Several GRL approaches are summarized and implemented in the proposed framework.
Results are analyzed in multiple perspectives and dimensions to compare the characteristic of different GRL approaches in intelligent transportation scenarios.
- Score: 12.34509371288594
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: An efficient and reliable multi-agent decision-making system is highly
demanded for the safe and efficient operation of connected autonomous vehicles
in intelligent transportation systems. Current researches mainly focus on the
Deep Reinforcement Learning (DRL) methods. However, utilizing DRL methods in
interactive traffic scenarios is hard to represent the mutual effects between
different vehicles and model the dynamic traffic environments due to the lack
of interactive information in the representation of the environments, which
results in low accuracy of cooperative decisions generation. To tackle these
difficulties, this research proposes a framework to enable different Graph
Reinforcement Learning (GRL) methods for decision-making, and compares their
performance in interactive driving scenarios. GRL methods combinate the Graph
Neural Network (GNN) and DRL to achieve the better decisions generation in
interactive scenarios of autonomous vehicles, where the features of interactive
scenarios are extracted by the GNN, and cooperative behaviors are generated by
DRL framework. Several GRL approaches are summarized and implemented in the
proposed framework. To evaluate the performance of the proposed GRL methods, an
interactive driving scenarios on highway with two ramps is constructed, and
simulated experiment in the SUMO platform is carried out to evaluate the
performance of different GRL approaches. Finally, results are analyzed in
multiple perspectives and dimensions to compare the characteristic of different
GRL approaches in intelligent transportation scenarios. Results show that the
implementation of GNN can well represents the interaction between vehicles, and
the combination of GNN and DRL is able to improve the performance of the
generation of lane-change behaviors. The source code of our work can be found
at https://github.com/Jacklinkk/TorchGRL.
Related papers
- End-to-end Driving in High-Interaction Traffic Scenarios with Reinforcement Learning [24.578178308010912]
We propose an end-to-end model-based RL algorithm named Ramble to address these issues.
By learning a dynamics model of the environment, Ramble can foresee upcoming traffic events and make more informed, strategic decisions.
Ramble achieves state-of-the-art performance regarding route completion rate and driving score on the CARLA Leaderboard 2.0, showcasing its effectiveness in managing complex and dynamic traffic situations.
arXiv Detail & Related papers (2024-10-03T06:45:59Z) - Generative AI for Deep Reinforcement Learning: Framework, Analysis, and Use Cases [60.30995339585003]
Deep reinforcement learning (DRL) has been widely applied across various fields and has achieved remarkable accomplishments.
DRL faces certain limitations, including low sample efficiency and poor generalization.
We present how to leverage generative AI (GAI) to address these issues and enhance the performance of DRL algorithms.
arXiv Detail & Related papers (2024-05-31T01:25:40Z) - Interactive Autonomous Navigation with Internal State Inference and
Interactivity Estimation [58.21683603243387]
We propose three auxiliary tasks with relational-temporal reasoning and integrate them into the standard Deep Learning framework.
These auxiliary tasks provide additional supervision signals to infer the behavior patterns other interactive agents.
Our approach achieves robust and state-of-the-art performance in terms of standard evaluation metrics.
arXiv Detail & Related papers (2023-11-27T18:57:42Z) - Action and Trajectory Planning for Urban Autonomous Driving with
Hierarchical Reinforcement Learning [1.3397650653650457]
We propose an action and trajectory planner using Hierarchical Reinforcement Learning (atHRL) method.
We empirically verify the efficacy of atHRL through extensive experiments in complex urban driving scenarios.
arXiv Detail & Related papers (2023-06-28T07:11:02Z) - MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion
Control in Real Networks [63.24965775030673]
We propose a novel Reinforcement Learning (RL) approach to design generic Congestion Control (CC) algorithms.
Our solution, MARLIN, uses the Soft Actor-Critic algorithm to maximize both entropy and return.
We trained MARLIN on a real network with varying background traffic patterns to overcome the sim-to-real mismatch.
arXiv Detail & Related papers (2023-02-02T18:27:20Z) - Hybrid Reinforcement Learning-Based Eco-Driving Strategy for Connected
and Automated Vehicles at Signalized Intersections [3.401874022426856]
Vision-perceptive methods are integrated with vehicle-to-infrastructure (V2I) communications to achieve higher mobility and energy efficiency.
HRL framework has three components: a rule-based driving manager that operates the collaboration between the rule-based policies and the RL policy.
Experiments show that our HRL method can reduce energy consumption by 12.70% and save 11.75% travel time when compared with a state-of-the-art model-based Eco-Driving approach.
arXiv Detail & Related papers (2022-01-19T19:31:12Z) - Pessimistic Model Selection for Offline Deep Reinforcement Learning [56.282483586473816]
Deep Reinforcement Learning (DRL) has demonstrated great potentials in solving sequential decision making problems in many applications.
One main barrier is the over-fitting issue that leads to poor generalizability of the policy learned by DRL.
We propose a pessimistic model selection (PMS) approach for offline DRL with a theoretical guarantee.
arXiv Detail & Related papers (2021-11-29T06:29:49Z) - Semantic-Aware Collaborative Deep Reinforcement Learning Over Wireless
Cellular Networks [82.02891936174221]
Collaborative deep reinforcement learning (CDRL) algorithms in which multiple agents can coordinate over a wireless network is a promising approach.
In this paper, a novel semantic-aware CDRL method is proposed to enable a group of untrained agents with semantically-linked DRL tasks to collaborate efficiently across a resource-constrained wireless cellular network.
arXiv Detail & Related papers (2021-11-23T18:24:47Z) - Vehicular Cooperative Perception Through Action Branching and Federated
Reinforcement Learning [101.64598586454571]
A novel framework is proposed to allow reinforcement learning-based vehicular association, resource block (RB) allocation, and content selection of cooperative perception messages (CPMs)
A federated RL approach is introduced in order to speed up the training process across vehicles.
Results show that federated RL improves the training process, where better policies can be achieved within the same amount of time compared to the non-federated approach.
arXiv Detail & Related papers (2020-12-07T02:09:15Z) - Pareto Deterministic Policy Gradients and Its Application in 5G Massive
MIMO Networks [32.099949375036495]
We consider jointly optimizing cell load balance and network throughput via a reinforcement learning (RL) approach.
Our rationale behind using RL is to circumvent the challenges of analytically modeling user mobility and network dynamics.
To accomplish this joint optimization, we integrate vector rewards into the RL value network and conduct RL action via a separate policy network.
arXiv Detail & Related papers (2020-12-02T15:35:35Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.