Formulation and validation of a car-following model based on deep reinforcement learning

• URL: http://arxiv.org/abs/2109.14268v1
• Date: Wed, 29 Sep 2021 08:27:12 GMT
• Title: Formulation and validation of a car-following model based on deep reinforcement learning
• Authors: Fabian Hart, Ostap Okhrin, Martin Treiber
• Abstract summary: We propose and validate a novel car following model based on deep reinforcement learning. Our model is trained to maximize externally given reward functions for the free and car-following regimes. The parameters of these reward functions resemble that of traditional models such as the Intelligent Driver Model.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose and validate a novel car following model based on deep reinforcement learning. Our model is trained to maximize externally given reward functions for the free and car-following regimes rather than reproducing existing follower trajectories. The parameters of these reward functions such as desired speed, time gap, or accelerations resemble that of traditional models such as the Intelligent Driver Model (IDM) and allow for explicitly implementing different driving styles. Moreover, they partially lift the black-box nature of conventional neural network models. The model is trained on leading speed profiles governed by a truncated Ornstein-Uhlenbeck process reflecting a realistic leader's kinematics. This allows for arbitrary driving situations and an infinite supply of training data. For various parameterizations of the reward functions, and for a wide variety of artificial and real leader data, the model turned out to be unconditionally string stable, comfortable, and crash-free. String stability has been tested with a platoon of five followers following an artificial and a real leading trajectory. A cross-comparison with the IDM calibrated to the goodness-of-fit of the relative gaps showed a higher reward compared to the traditional model and a better goodness-of-fit.

Related papers

• MetaFollower: Adaptable Personalized Autonomous Car Following [63.90050686330677]
  We propose an adaptable personalized car-following framework - MetaFollower. We first utilize Model-Agnostic Meta-Learning (MAML) to extract common driving knowledge from various CF events. We additionally combine Long Short-Term Memory (LSTM) and Intelligent Driver Model (IDM) to reflect temporal heterogeneity with high interpretability.
  arXiv  Detail & Related papers  (2024-06-23T15:30:40Z)
• LORD: Large Models based Opposite Reward Design for Autonomous Driving [11.717821043996352]
  We introduce LORD, a novel large models based opposite reward design through undesired linguistic goals. Our proposed framework shows its efficiency in leveraging the power of large pretrained models for achieving safe and enhanced autonomous driving.
  arXiv  Detail & Related papers  (2024-03-27T19:30:06Z)
• Secrets of RLHF in Large Language Models Part II: Reward Modeling [134.97964938009588]
  We introduce a series of novel methods to mitigate the influence of incorrect and ambiguous preferences in the dataset. We also introduce contrastive learning to enhance the ability of reward models to distinguish between chosen and rejected responses.
  arXiv  Detail & Related papers  (2024-01-11T17:56:59Z)
• A Tricycle Model to Accurately Control an Autonomous Racecar with Locked Differential [71.53284767149685]
  We present a novel formulation to model the effects of a locked differential on the lateral dynamics of an autonomous open-wheel racecar. We include a micro-steps discretization approach to accurately linearize the dynamics and produce a prediction suitable for real-time implementation.
  arXiv  Detail & Related papers  (2023-12-22T16:29:55Z)
• RACER: Rational Artificial Intelligence Car-following-model Enhanced by Reality [51.244807332133696]
  This paper introduces RACER, a cutting-edge deep learning car-following model to predict Adaptive Cruise Control (ACC) driving behavior. Unlike conventional models, RACER effectively integrates Rational Driving Constraints (RDCs), crucial tenets of actual driving. RACER excels across key metrics, such as acceleration, velocity, and spacing, registering zero violations.
  arXiv  Detail & Related papers  (2023-12-12T06:21:30Z)
• EnsembleFollower: A Hybrid Car-Following Framework Based On Reinforcement Learning and Hierarchical Planning [22.63087292154406]
  We propose a hierarchical planning framework for achieving advanced human-like car-following. The EnsembleFollower framework involves a high-level Reinforcement Learning-based agent responsible for judiciously managing multiple low-level car-following models. We evaluate the proposed method based on real-world driving data from the HighD dataset.
  arXiv  Detail & Related papers  (2023-08-30T12:55:02Z)
• Value function estimation using conditional diffusion models for control [62.27184818047923]
  We propose a simple algorithm called Diffused Value Function (DVF) It learns a joint multi-step model of the environment-robot interaction dynamics using a diffusion model. We show how DVF can be used to efficiently capture the state visitation measure for multiple controllers.
  arXiv  Detail & Related papers  (2023-06-09T18:40:55Z)
• Sense, Imagine, Act: Multimodal Perception Improves Model-Based Reinforcement Learning for Head-to-Head Autonomous Racing [10.309579267966361]
  Model-based reinforcement learning (MBRL) techniques have recently yielded promising results for real-world autonomous racing. This paper proposes a self-supervised sensor fusion technique that combines egocentric LiDAR and RGB camera observations collected from the F1TENTH Gym. The resulting Dreamer agent safely avoided collisions and won the most races compared to other tested baselines in zero-shot head-to-head autonomous racing.
  arXiv  Detail & Related papers  (2023-05-08T14:49:02Z)
• An active inference model of car following: Advantages and applications [6.905724739762358]
  Driver process models play a central role in the testing, verification, and development of automated and autonomous vehicle technologies. Data-driven machine learning models are more capable than rule-based models but are limited by the need for large training datasets and their lack of interpretability. We propose a novel car following modeling approach using active inference, which has comparable behavioral flexibility to data-driven models while maintaining interpretability.
  arXiv  Detail & Related papers  (2023-03-27T13:39:26Z)
• Your Autoregressive Generative Model Can be Better If You Treat It as an Energy-Based One [83.5162421521224]
  We propose a unique method termed E-ARM for training autoregressive generative models. E-ARM takes advantage of a well-designed energy-based learning objective. We show that E-ARM can be trained efficiently and is capable of alleviating the exposure bias problem.
  arXiv  Detail & Related papers  (2022-06-26T10:58:41Z)
• Iterative Semi-parametric Dynamics Model Learning For Autonomous Racing [2.40966076588569]
  We develop and apply an iterative learning semi-parametric model, with a neural network, to the task of autonomous racing. We show that our model can learn more accurately than a purely parametric model and generalize better than a purely non-parametric model.
  arXiv  Detail & Related papers  (2020-11-17T16:24:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.