Robust Longitudinal Control for Vehicular Autonomous Platoons Using Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2206.01175v1
• Date: Tue, 31 May 2022 20:38:12 GMT
• Title: Robust Longitudinal Control for Vehicular Autonomous Platoons Using Deep Reinforcement Learning
• Authors: Armando Alves Neto and Leonardo Amaral Mozelli
• Abstract summary: We propose an approach to generalize the training process of a vehicular platoon, such that the acceleration command of each agent becomes independent of the network topology. We illustrate the effectiveness of our proposal with experiments using different network topologies, uncertain parameters, and external forces.
• Score: 3.0552168294716298
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the last few years, researchers have applied machine learning strategies in the context of vehicular platoons to increase the safety and efficiency of cooperative transportation. Reinforcement Learning methods have been employed in the longitudinal spacing control of Cooperative Adaptive Cruise Control systems, but to date, none of those studies have addressed problems of disturbance rejection in such scenarios. Characteristics such as uncertain parameters in the model and external interferences may prevent agents from reaching null-spacing errors when traveling at cruising speed. On the other hand, complex communication topologies lead to specific training processes that can not be generalized to other contexts, demanding re-training every time the configuration changes. Therefore, in this paper, we propose an approach to generalize the training process of a vehicular platoon, such that the acceleration command of each agent becomes independent of the network topology. Also, we have modeled the acceleration input as a term with integral action, such that the Convolutional Neural Network is capable of learning corrective actions when the states are disturbed by unknown effects. We illustrate the effectiveness of our proposal with experiments using different network topologies, uncertain parameters, and external forces. Comparative analyses, in terms of the steady-state error and overshoot response, were conducted against the state-of-the-art literature. The findings offer new insights concerning generalization and robustness of using Reinforcement Learning in the control of autonomous platoons.

Related papers

• World Models for Anomaly Detection during Model-Based Reinforcement Learning Inference [3.591122855617648]
  Learning-based controllers are often purposefully kept out of real-world applications due to concerns about their safety and reliability. We explore how state-of-the-art world models in Model-Based Reinforcement Learning can be utilized beyond the training phase to ensure a deployed policy only operates within regions of the state-space it is sufficiently familiar with.
  arXiv  Detail & Related papers  (2025-03-04T12:25:01Z)
• End-to-End Steering for Autonomous Vehicles via Conditional Imitation Co-Learning [1.5020330976600735]
  This work introduces the conditional imitation co-learning (CIC) approach to address this issue. We propose posing the steering regression problem as classification, we use a classification-regression hybrid loss to bridge the gap between regression and classification. Our model is demonstrated to improve autonomous driving success rate in unseen environment by 62% on average compared to the CIL method.
  arXiv  Detail & Related papers  (2024-11-25T06:37:48Z)
• Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Models [60.87795376541144]
  A world model is a neural network capable of predicting an agent's next state given past states and actions. During end-to-end training, our policy learns how to recover from errors by aligning with states observed in human demonstrations. We present qualitative and quantitative results, demonstrating significant improvements upon prior state of the art in closed-loop testing.
  arXiv  Detail & Related papers  (2024-09-25T06:48:25Z)
• Interactive Autonomous Navigation with Internal State Inference and Interactivity Estimation [58.21683603243387]
  We propose three auxiliary tasks with relational-temporal reasoning and integrate them into the standard Deep Learning framework. These auxiliary tasks provide additional supervision signals to infer the behavior patterns other interactive agents. Our approach achieves robust and state-of-the-art performance in terms of standard evaluation metrics.
  arXiv  Detail & Related papers  (2023-11-27T18:57:42Z)
• DTC: Deep Tracking Control [16.2850135844455]
  We propose a hybrid control architecture that combines the advantages of both worlds to achieve greater robustness, foot-placement accuracy, and terrain generalization. A deep neural network policy is trained in simulation, aiming to track the optimized footholds. We demonstrate superior robustness in the presence of slippery or deformable ground when compared to model-based counterparts.
  arXiv  Detail & Related papers  (2023-09-27T07:57:37Z)
• Learning Dynamics and Generalization in Reinforcement Learning [59.530058000689884]
  We show theoretically that temporal difference learning encourages agents to fit non-smooth components of the value function early in training. We show that neural networks trained using temporal difference algorithms on dense reward tasks exhibit weaker generalization between states than randomly networks and gradient networks trained with policy methods.
  arXiv  Detail & Related papers  (2022-06-05T08:49:16Z)
• Coach-assisted Multi-Agent Reinforcement Learning Framework for Unexpected Crashed Agents [120.91291581594773]
  We present a formal formulation of a cooperative multi-agent reinforcement learning system with unexpected crashes. We propose a coach-assisted multi-agent reinforcement learning framework, which introduces a virtual coach agent to adjust the crash rate during training. To the best of our knowledge, this work is the first to study the unexpected crashes in the multi-agent system.
  arXiv  Detail & Related papers  (2022-03-16T08:22:45Z)
• The Impact of Missing Velocity Information in Dynamic Obstacle Avoidance based on Deep Reinforcement Learning [0.0]
  We introduce a novel approach to dynamic obstacle avoidance based on Deep Reinforcement Learning. We thoroughly investigate the effect of missing velocity information on an agent's performance in obstacle avoidance tasks.
  arXiv  Detail & Related papers  (2021-12-23T11:07:00Z)
• Trajectory Tracking of Underactuated Sea Vessels With Uncertain Dynamics: An Integral Reinforcement Learning Approach [2.064612766965483]
  An online machine learning mechanism based on integral reinforcement learning is proposed to find a solution for a class of nonlinear tracking problems. The solution is implemented using an online value iteration process which is realized by employing means of the adaptive critics and gradient descent approaches.
  arXiv  Detail & Related papers  (2021-04-01T01:41:49Z)
• Adversarial Training is Not Ready for Robot Learning [55.493354071227174]
  Adversarial training is an effective method to train deep learning models that are resilient to norm-bounded perturbations. We show theoretically and experimentally that neural controllers obtained via adversarial training are subjected to three types of defects. Our results suggest that adversarial training is not yet ready for robot learning.
  arXiv  Detail & Related papers  (2021-03-15T07:51:31Z)
• Reinforcement Learning for Autonomous Driving with Latent State Inference and Spatial-Temporal Relationships [46.965260791099986]
  We show that explicitly inferring the latent state and encoding spatial-temporal relationships in a reinforcement learning framework can help address this difficulty. We encode prior knowledge on the latent states of other drivers through a framework that combines the reinforcement learner with a supervised learner. The proposed framework significantly improves performance in the context of navigating T-intersections compared with state-of-the-art baseline approaches.
  arXiv  Detail & Related papers  (2020-11-09T08:55:12Z)
• Enhanced Adversarial Strategically-Timed Attacks against Deep Reinforcement Learning [91.13113161754022]
  We introduce timing-based adversarial strategies against a DRL-based navigation system by jamming in physical noise patterns on the selected time frames. Our experimental results show that the adversarial timing attacks can lead to a significant performance drop.
  arXiv  Detail & Related papers  (2020-02-20T21:39:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.