Progression Cognition Reinforcement Learning with Prioritized Experience
for Multi-Vehicle Pursuit
- URL: http://arxiv.org/abs/2306.05016v1
- Date: Thu, 8 Jun 2023 08:10:46 GMT
- Title: Progression Cognition Reinforcement Learning with Prioritized Experience
for Multi-Vehicle Pursuit
- Authors: Xinhang Li, Yiying Yang, Zheng Yuan, Zhe Wang, Qinwen Wang, Chen Xu,
Lei Li, Jianhua He and Lin Zhang
- Abstract summary: This paper proposes a Cognition Reinforcement Learning with Prioritized Experience for MVP in urban traffic scenes.
PEPCRL-MVP uses a prioritization network to assess the transitions in the global experience replay buffer according to the parameters of each MARL agent.
PEPCRL-MVP improves pursuing efficiency by 3.95% over TD3-DMAP and its success rate is 34.78% higher than that of MADDPG.
- Score: 19.00359253910912
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Multi-vehicle pursuit (MVP) such as autonomous police vehicles pursuing
suspects is important but very challenging due to its mission and safety
critical nature. While multi-agent reinforcement learning (MARL) algorithms
have been proposed for MVP problem in structured grid-pattern roads, the
existing algorithms use randomly training samples in centralized learning,
which leads to homogeneous agents showing low collaboration performance. For
the more challenging problem of pursuing multiple evading vehicles, these
algorithms typically select a fixed target evading vehicle for pursuing
vehicles without considering dynamic traffic situation, which significantly
reduces pursuing success rate. To address the above problems, this paper
proposes a Progression Cognition Reinforcement Learning with Prioritized
Experience for MVP (PEPCRL-MVP) in urban multi-intersection dynamic traffic
scenes. PEPCRL-MVP uses a prioritization network to assess the transitions in
the global experience replay buffer according to the parameters of each MARL
agent. With the personalized and prioritized experience set selected via the
prioritization network, diversity is introduced to the learning process of
MARL, which can improve collaboration and task related performance.
Furthermore, PEPCRL-MVP employs an attention module to extract critical
features from complex urban traffic environments. These features are used to
develop progression cognition method to adaptively group pursuing vehicles.
Each group efficiently target one evading vehicle in dynamic driving
environments. Extensive experiments conducted with a simulator over
unstructured roads of an urban area show that PEPCRL-MVP is superior to other
state-of-the-art methods. Specifically, PEPCRL-MVP improves pursuing efficiency
by 3.95% over TD3-DMAP and its success rate is 34.78% higher than that of
MADDPG. Codes are open sourced.
Related papers
- Action and Trajectory Planning for Urban Autonomous Driving with
Hierarchical Reinforcement Learning [1.3397650653650457]
We propose an action and trajectory planner using Hierarchical Reinforcement Learning (atHRL) method.
We empirically verify the efficacy of atHRL through extensive experiments in complex urban driving scenarios.
arXiv Detail & Related papers (2023-06-28T07:11:02Z) - Graded-Q Reinforcement Learning with Information-Enhanced State Encoder
for Hierarchical Collaborative Multi-Vehicle Pursuit [11.195170949292496]
The multi-vehicle pursuit (MVP) is becoming a hot research topic in Intelligent Transportation System (ITS)
This paper proposed a graded-Q reinforcement learning with information-enhanced state encoder (GQRL-IESE) framework to address this hierarchical collaborative pursuit problem.
In the GQRL-IESE, a cooperative graded Q scheme is proposed to facilitate the decision-making of pursuing vehicles to improve pursuing efficiency.
arXiv Detail & Related papers (2022-10-24T16:35:34Z) - CLUTR: Curriculum Learning via Unsupervised Task Representation Learning [130.79246770546413]
CLUTR is a novel curriculum learning algorithm that decouples task representation and curriculum learning into a two-stage optimization.
We show CLUTR outperforms PAIRED, a principled and popular UED method, in terms of generalization and sample efficiency in the challenging CarRacing and navigation environments.
arXiv Detail & Related papers (2022-10-19T01:45:29Z) - Unified Automatic Control of Vehicular Systems with Reinforcement
Learning [64.63619662693068]
This article contributes a streamlined methodology for vehicular microsimulation.
It discovers high performance control strategies with minimal manual design.
The study reveals numerous emergent behaviors resembling wave mitigation, traffic signaling, and ramp metering.
arXiv Detail & Related papers (2022-07-30T16:23:45Z) - Learning energy-efficient driving behaviors by imitating experts [75.12960180185105]
This paper examines the role of imitation learning in bridging the gap between control strategies and realistic limitations in communication and sensing.
We show that imitation learning can succeed in deriving policies that, if adopted by 5% of vehicles, may boost the energy-efficiency of networks with varying traffic conditions by 15% using only local observations.
arXiv Detail & Related papers (2022-06-28T17:08:31Z) - AI-aided Traffic Control Scheme for M2M Communications in the Internet
of Vehicles [61.21359293642559]
The dynamics of traffic and the heterogeneous requirements of different IoV applications are not considered in most existing studies.
We consider a hybrid traffic control scheme and use proximal policy optimization (PPO) method to tackle it.
arXiv Detail & Related papers (2022-03-05T10:54:05Z) - A Deep Reinforcement Learning Approach for Traffic Signal Control
Optimization [14.455497228170646]
Inefficient traffic signal control methods may cause numerous problems, such as traffic congestion and waste of energy.
This paper first proposes a multi-agent deep deterministic policy gradient (MADDPG) method by extending the actor-critic policy gradient algorithms.
arXiv Detail & Related papers (2021-07-13T14:11:04Z) - MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Control [54.162449208797334]
Traffic signal control aims to coordinate traffic signals across intersections to improve the traffic efficiency of a district or a city.
Deep reinforcement learning (RL) has been applied to traffic signal control recently and demonstrated promising performance where each traffic signal is regarded as an agent.
We propose a novel Meta Variationally Intrinsic Motivated (MetaVIM) RL method to learn the decentralized policy for each intersection that considers neighbor information in a latent way.
arXiv Detail & Related papers (2021-01-04T03:06:08Z) - Detecting 32 Pedestrian Attributes for Autonomous Vehicles [103.87351701138554]
In this paper, we address the problem of jointly detecting pedestrians and recognizing 32 pedestrian attributes.
We introduce a Multi-Task Learning (MTL) model relying on a composite field framework, which achieves both goals in an efficient way.
We show competitive detection and attribute recognition results, as well as a more stable MTL training.
arXiv Detail & Related papers (2020-12-04T15:10:12Z) - Decision-making for Autonomous Vehicles on Highway: Deep Reinforcement
Learning with Continuous Action Horizon [14.059728921828938]
This paper utilizes the deep reinforcement learning (DRL) method to address the continuous-horizon decision-making problem on the highway.
The running objective of the ego automated vehicle is to execute an efficient and smooth policy without collision.
The PPO-DRL-based decision-making strategy is estimated from multiple perspectives, including the optimality, learning efficiency, and adaptability.
arXiv Detail & Related papers (2020-08-26T22:49:27Z) - Efficient Sampling-Based Maximum Entropy Inverse Reinforcement Learning
with Application to Autonomous Driving [35.44498286245894]
We present an efficient sampling-based maximum-entropy inverse reinforcement learning (IRL) algorithm in this paper.
We evaluate the proposed algorithm on real driving data, including both non-interactive and interactive scenarios.
arXiv Detail & Related papers (2020-06-22T01:41:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.