Deep Reinforcement Learning Aided Platoon Control Relying on V2X Information

• URL: http://arxiv.org/abs/2203.15781v1
• Date: Mon, 28 Mar 2022 02:11:54 GMT
• Title: Deep Reinforcement Learning Aided Platoon Control Relying on V2X Information
• Authors: Lei Lei, Tong Liu, Kan Zheng and Lajos Hanzo
• Abstract summary: The impact of Vehicle-to-Everything (V2X) communications on platoon control performance is investigated. Our objective is to find the specific set of information that should be shared among the vehicles for the construction of the most appropriate state space. More meritorious information is given higher priority in transmission, since including it in the state space has a higher probability in offsetting the negative effect of having higher state dimensions.
• Score: 78.18186960475974
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The impact of Vehicle-to-Everything (V2X) communications on platoon control performance is investigated. Platoon control is essentially a sequential stochastic decision problem (SSDP), which can be solved by Deep Reinforcement Learning (DRL) to deal with both the control constraints and uncertainty in the platoon leading vehicle's behavior. In this context, the value of V2X communications for DRL-based platoon controllers is studied with an emphasis on the tradeoff between the gain of including exogenous information in the system state for reducing uncertainty and the performance erosion due to the curse-of-dimensionality. Our objective is to find the specific set of information that should be shared among the vehicles for the construction of the most appropriate state space. SSDP models are conceived for platoon control under different information topologies (IFT) by taking into account `just sufficient' information. Furthermore, theorems are established for comparing the performance of their optimal policies. In order to determine whether a piece of information should or should not be transmitted for improving the DRL-based control policy, we quantify its value by deriving the conditional KL divergence of the transition models. More meritorious information is given higher priority in transmission, since including it in the state space has a higher probability in offsetting the negative effect of having higher state dimensions. Finally, simulation results are provided to illustrate the theoretical analysis.

Related papers

• Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining [74.83412846804977]
  Reinforcement learning (RL)-based fine-tuning has become a crucial step in post-training language models. We present a systematic end-to-end study of RL fine-tuning for mathematical reasoning by training models entirely from scratch.
  arXiv  Detail & Related papers  (2025-04-10T17:15:53Z)
• Multi-Timescale Control and Communications with Deep Reinforcement Learning -- Part I: Communication-Aware Vehicle Control [15.390800228536536]
  We propose a joint optimization framework of multi-timescale control and communications based on Deep Reinforcement Learning (DRL) In this paper (Part I), we first decompose the problem into a communication-aware DRL-based PC sub-problem and a control-aware DRL-based RRA sub-problem. To improve the PC performance under random observation delay, the PC state space is augmented with the observation delay and PC action history. It is proved that the optimal policy for the augmented state MDP is optimal for the original PC problem with observation delay.
  arXiv  Detail & Related papers  (2023-11-19T09:51:58Z)
• Optimal Transport-Guided Conditional Score-Based Diffusion Models [63.14903268958398]
  Conditional score-based diffusion model (SBDM) is for conditional generation of target data with paired data as condition, and has achieved great success in image translation. To tackle the applications with partially paired or even unpaired dataset, we propose a novel Optimal Transport-guided Conditional Score-based diffusion model (OTCS) in this paper.
  arXiv  Detail & Related papers  (2023-11-02T13:28:44Z)
• Efficient Reinforcement Learning with Impaired Observability: Learning to Act with Delayed and Missing State Observations [92.25604137490168]
  This paper introduces a theoretical investigation into efficient reinforcement learning in control systems. We present algorithms and establish near-optimal regret upper and lower bounds, of the form $tildemathcalO(sqrtrm poly(H) SAK)$, for RL in the delayed and missing observation settings.
  arXiv  Detail & Related papers  (2023-06-02T02:46:39Z)
• Efficient Deep Reinforcement Learning Requires Regulating Overfitting [91.88004732618381]
  We show that high temporal-difference (TD) error on the validation set of transitions is the main culprit that severely affects the performance of deep RL algorithms. We show that a simple online model selection method that targets the validation TD error is effective across state-based DMC and Gym tasks.
  arXiv  Detail & Related papers  (2023-04-20T17:11:05Z)
• Development of a CAV-based Intersection Control System and Corridor Level Impact Assessment [0.696125353550498]
  This paper presents a signal-free intersection control system for CAVs by combination of a pixel reservation algorithm and a Deep Reinforcement Learning (DRL) decision-making logic. The proposed model reduces delay by 50%, 29%, and 23% in moderate, high, and extreme volume regimes compared to the other CAV-based control system.
  arXiv  Detail & Related papers  (2022-08-21T21:56:20Z)
• Steady-State Error Compensation in Reference Tracking and Disturbance Rejection Problems for Reinforcement Learning-Based Control [0.9023847175654602]
  Reinforcement learning (RL) is a promising, upcoming topic in automatic control applications. Initiative action state augmentation (IASA) for actor-critic-based RL controllers is introduced. This augmentation does not require any expert knowledge, leaving the approach model free.
  arXiv  Detail & Related papers  (2022-01-31T16:29:19Z)
• Attacking Deep Reinforcement Learning-Based Traffic Signal Control Systems with Colluding Vehicles [4.2455052426413085]
  This paper formulates a novel task in which a group of vehicles can cooperatively send falsified information to "cheat" DRL-based ATCS. CollusionVeh is a generic and effective vehicle-colluding framework composed of a road situation encoder, a vehicle interpreter, and a communication mechanism. The research outcomes could help improve the reliability and robustness of the ATCS and better protect the smart mobility systems.
  arXiv  Detail & Related papers  (2021-11-04T13:10:33Z)
• How Are Learned Perception-Based Controllers Impacted by the Limits of Robust Control? [17.775878968489852]
  We revisit the difficulty of optimal control problems in terms of system properties like minimum eigenvalues of controllability/observability gramians. We ask: to what extent are quantifiable control and perceptual difficulty metrics of a task predictive of the performance and sample complexity of data-driven controllers? Our results show that the fundamental limits of robust control have corresponding implications for the sample-efficiency and performance of learned perception-based controllers.
  arXiv  Detail & Related papers  (2021-04-02T00:31:31Z)
• Vehicular Cooperative Perception Through Action Branching and Federated Reinforcement Learning [101.64598586454571]
  A novel framework is proposed to allow reinforcement learning-based vehicular association, resource block (RB) allocation, and content selection of cooperative perception messages (CPMs) A federated RL approach is introduced in order to speed up the training process across vehicles. Results show that federated RL improves the training process, where better policies can be achieved within the same amount of time compared to the non-federated approach.
  arXiv  Detail & Related papers  (2020-12-07T02:09:15Z)
• How Training Data Impacts Performance in Learning-based Control [67.7875109298865]
  This paper derives an analytical relationship between the density of the training data and the control performance. We formulate a quality measure for the data set, which we refer to as $rho$-gap. We show how the $rho$-gap can be applied to a feedback linearizing control law.
  arXiv  Detail & Related papers  (2020-05-25T12:13:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.