Implicit Sensing in Traffic Optimization: Advanced Deep Reinforcement Learning Techniques

• URL: http://arxiv.org/abs/2309.14395v1
• Date: Mon, 25 Sep 2023 15:33:08 GMT
• Title: Implicit Sensing in Traffic Optimization: Advanced Deep Reinforcement Learning Techniques
• Authors: Emanuel Figetakis, Yahuza Bello, Ahmed Refaey, Lei Lei, Medhat Moussa
• Abstract summary: We present an integrated car-following and lane-changing decision-control system based on Deep Reinforcement Learning (DRL) We employ the well-known DQN algorithm to train the RL agent to make the appropriate decision accordingly. We evaluate the performance of the proposed model under two policies; epsilon-greedy policy and Boltzmann policy.
• Score: 4.042717292629285
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A sudden roadblock on highways due to many reasons such as road maintenance, accidents, and car repair is a common situation we encounter almost daily. Autonomous Vehicles (AVs) equipped with sensors that can acquire vehicle dynamics such as speed, acceleration, and location can make intelligent decisions to change lanes before reaching a roadblock. A number of literature studies have examined car-following models and lane-changing models. However, only a few studies proposed an integrated car-following and lane-changing model, which has the potential to model practical driving maneuvers. Hence, in this paper, we present an integrated car-following and lane-changing decision-control system based on Deep Reinforcement Learning (DRL) to address this issue. Specifically, we consider a scenario where sudden construction work will be carried out along a highway. We model the scenario as a Markov Decision Process (MDP) and employ the well-known DQN algorithm to train the RL agent to make the appropriate decision accordingly (i.e., either stay in the same lane or change lanes). To overcome the delay and computational requirement of DRL algorithms, we adopt an MEC-assisted architecture where the RL agents are trained on MEC servers. We utilize the highly reputable SUMO simulator and OPENAI GYM to evaluate the performance of the proposed model under two policies; {\epsilon}-greedy policy and Boltzmann policy. The results unequivocally demonstrate that the DQN agent trained using the {\epsilon}-greedy policy significantly outperforms the one trained with the Boltzmann policy.

Related papers

• Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Models [60.87795376541144]
  A world model is a neural network capable of predicting an agent's next state given past states and actions. During end-to-end training, our policy learns how to recover from errors by aligning with states observed in human demonstrations. We present qualitative and quantitative results, demonstrating significant improvements upon prior state of the art in closed-loop testing.
  arXiv  Detail & Related papers  (2024-09-25T06:48:25Z)
• HighwayLLM: Decision-Making and Navigation in Highway Driving with RL-Informed Language Model [5.4854443795779355]
  This study presents a novel approach, HighwayLLM, which harnesses the reasoning capabilities of large language models (LLMs) to predict the future waypoints for ego-vehicle's navigation. Our approach also utilizes a pre-trained Reinforcement Learning (RL) model to serve as a high-level planner, making decisions on appropriate meta-level actions.
  arXiv  Detail & Related papers  (2024-05-22T11:32:37Z)
• DRNet: A Decision-Making Method for Autonomous Lane Changingwith Deep Reinforcement Learning [7.2282857478457805]
  "DRNet" is a novel DRL-based framework that enables a DRL agent to learn to drive by executing reasonable lane changing on simulated highways. Our DRL agent has the ability to learn the desired task without causing collisions and outperforms DDQN and other baseline models.
  arXiv  Detail & Related papers  (2023-11-02T21:17:52Z)
• Rethinking Closed-loop Training for Autonomous Driving [82.61418945804544]
  We present the first empirical study which analyzes the effects of different training benchmark designs on the success of learning agents. We propose trajectory value learning (TRAVL), an RL-based driving agent that performs planning with multistep look-ahead. Our experiments show that TRAVL can learn much faster and produce safer maneuvers compared to all the baselines.
  arXiv  Detail & Related papers  (2023-06-27T17:58:39Z)
• Comprehensive Training and Evaluation on Deep Reinforcement Learning for Automated Driving in Various Simulated Driving Maneuvers [0.4241054493737716]
  This study implements, evaluating, and comparing the two DRL algorithms, Deep Q-networks (DQN) and Trust Region Policy Optimization (TRPO) Models trained on the designed ComplexRoads environment can adapt well to other driving maneuvers with promising overall performance.
  arXiv  Detail & Related papers  (2023-06-20T11:41:01Z)
• FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing [71.76084256567599]
  We present a system that enables an autonomous small-scale RC car to drive aggressively from visual observations using reinforcement learning (RL) Our system, FastRLAP (faster lap), trains autonomously in the real world, without human interventions, and without requiring any simulation or expert demonstrations. The resulting policies exhibit emergent aggressive driving skills, such as timing braking and acceleration around turns and avoiding areas which impede the robot's motion, approaching the performance of a human driver using a similar first-person interface over the course of training.
  arXiv  Detail & Related papers  (2023-04-19T17:33:47Z)
• Prediction Based Decision Making for Autonomous Highway Driving [3.6818636539023175]
  This paper proposes a Prediction-based Deep Reinforcement Learning (PDRL) decision-making model. It considers the manoeuvre intentions of surrounding vehicles in the decision-making process for highway driving. The results show that the proposed PDRL model improves the decision-making performance compared to a Deep Reinforcement Learning (DRL) model by decreasing collision numbers.
  arXiv  Detail & Related papers  (2022-09-05T19:28:30Z)
• Verifying Learning-Based Robotic Navigation Systems [61.01217374879221]
  We show how modern verification engines can be used for effective model selection. Specifically, we use verification to detect and rule out policies that may demonstrate suboptimal behavior. Our work is the first to demonstrate the use of verification backends for recognizing suboptimal DRL policies in real-world robots.
  arXiv  Detail & Related papers  (2022-05-26T17:56:43Z)
• Causal Imitative Model for Autonomous Driving [85.78593682732836]
  We propose Causal Imitative Model (CIM) to address inertia and collision problems. CIM explicitly discovers the causal model and utilizes it to train the policy. Our experiments show that our method outperforms previous work in terms of inertia and collision rates.
  arXiv  Detail & Related papers  (2021-12-07T18:59:15Z)
• Combining Reinforcement Learning with Model Predictive Control for On-Ramp Merging [10.480121529429631]
  Two broad classes of techniques have been proposed to solve motion planning problems in autonomous driving: Model Predictive Control (MPC) and Reinforcement Learning (RL) We first establish the strengths and weaknesses of state-of-the-art MPC and RL-based techniques through simulations. We subsequently present an algorithm which blends the model-free RL agent with the MPC solution and show that it provides better trade-offs between all metrics -- passenger comfort, efficiency, crash rate and robustness.
  arXiv  Detail & Related papers  (2020-11-17T07:42:11Z)
• Intelligent Roundabout Insertion using Deep Reinforcement Learning [68.8204255655161]
  We present a maneuver planning module able to negotiate the entering in busy roundabouts. The proposed module is based on a neural network trained to predict when and how entering the roundabout throughout the whole duration of the maneuver.
  arXiv  Detail & Related papers  (2020-01-03T11:16:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.