Improving Deep Dynamics Models for Autonomous Vehicles with Multimodal Latent Mapping of Surfaces

• URL: http://arxiv.org/abs/2303.11756v1
• Date: Tue, 21 Mar 2023 11:21:31 GMT
• Title: Improving Deep Dynamics Models for Autonomous Vehicles with Multimodal Latent Mapping of Surfaces
• Authors: Johan Vertens, Nicolai Dorka, Tim Welschehold, Michael Thompson, Wolfram Burgard
• Abstract summary: We propose a new approach that learns a surface-aware dynamics model by conditioning it on a latent variable vector. A latent mapper is trained to update these latent variables during inference from multiple modalities. We show that by using this model, the driving performance can be improved on varying and challenging surfaces.
• Score: 23.023397401781757
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The safe deployment of autonomous vehicles relies on their ability to effectively react to environmental changes. This can require maneuvering on varying surfaces which is still a difficult problem, especially for slippery terrains. To address this issue we propose a new approach that learns a surface-aware dynamics model by conditioning it on a latent variable vector storing surface information about the current location. A latent mapper is trained to update these latent variables during inference from multiple modalities on every traversal of the corresponding locations and stores them in a map. By training everything end-to-end with the loss of the dynamics model, we enforce the latent mapper to learn an update rule for the latent map that is useful for the subsequent dynamics model. We implement and evaluate our approach on a real miniature electric car. The results show that the latent map is updated to allow more accurate predictions of the dynamics model compared to a model without this information. We further show that by using this model, the driving performance can be improved on varying and challenging surfaces.

Related papers

• Characterized Diffusion Networks for Enhanced Autonomous Driving Trajectory Prediction [0.6202955567445396]
  We present a novel trajectory prediction model for autonomous driving. Our model enhances the accuracy and reliability of trajectory predictions by incorporating uncertainty estimation and complex agent interactions. The proposed model showcases strong potential for application in real-world autonomous driving systems.
  arXiv  Detail & Related papers  (2024-11-25T15:03:44Z)
• Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Models [60.87795376541144]
  A world model is a neural network capable of predicting an agent's next state given past states and actions. During end-to-end training, our policy learns how to recover from errors by aligning with states observed in human demonstrations. We present qualitative and quantitative results, demonstrating significant improvements upon prior state of the art in closed-loop testing.
  arXiv  Detail & Related papers  (2024-09-25T06:48:25Z)
• Continual Learning for Adaptable Car-Following in Dynamic Traffic Environments [16.587883982785]
  The continual evolution of autonomous driving technology requires car-following models that can adapt to diverse and dynamic traffic environments. Traditional learning-based models often suffer from performance degradation when encountering unseen traffic patterns due to a lack of continual learning capabilities. This paper proposes a novel car-following model based on continual learning that addresses this limitation.
  arXiv  Detail & Related papers  (2024-07-17T06:32:52Z)
• Planning with Adaptive World Models for Autonomous Driving [50.4439896514353]
  Motion planners (MPs) are crucial for safe navigation in complex urban environments. nuPlan, a recently released MP benchmark, addresses this limitation by augmenting real-world driving logs with closed-loop simulation logic. We present AdaptiveDriver, a model-predictive control (MPC) based planner that unrolls different world models conditioned on BehaviorNet's predictions.
  arXiv  Detail & Related papers  (2024-06-15T18:53:45Z)
• Guiding Attention in End-to-End Driving Models [49.762868784033785]
  Vision-based end-to-end driving models trained by imitation learning can lead to affordable solutions for autonomous driving. We study how to guide the attention of these models to improve their driving quality by adding a loss term during training. In contrast to previous work, our method does not require these salient semantic maps to be available during testing time.
  arXiv  Detail & Related papers  (2024-04-30T23:18:51Z)
• Trajeglish: Traffic Modeling as Next-Token Prediction [67.28197954427638]
  A longstanding challenge for self-driving development is simulating dynamic driving scenarios seeded from recorded driving logs. We apply tools from discrete sequence modeling to model how vehicles, pedestrians and cyclists interact in driving scenarios. Our model tops the Sim Agents Benchmark, surpassing prior work along the realism meta metric by 3.3% and along the interaction metric by 9.9%.
  arXiv  Detail & Related papers  (2023-12-07T18:53:27Z)
• Exploring Model Transferability through the Lens of Potential Energy [78.60851825944212]
  Transfer learning has become crucial in computer vision tasks due to the vast availability of pre-trained deep learning models. Existing methods for measuring the transferability of pre-trained models rely on statistical correlations between encoded static features and task labels. We present an insightful physics-inspired approach named PED to address these challenges.
  arXiv  Detail & Related papers  (2023-08-29T07:15:57Z)
• Context-Conditional Navigation with a Learning-Based Terrain- and Robot-Aware Dynamics Model [11.800678688260081]
  We develop a novel probabilistic, terrain- and robot-aware forward dynamics model, termed TRADYN. We evaluate our method in a simulated 2D navigation setting with a unicycle-like robot and different terrain layouts with spatially varying friction coefficients.
  arXiv  Detail & Related papers  (2023-07-18T12:42:59Z)
• TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction [149.5716746789134]
  We show data-driven traffic simulation can be formulated as a world model. We present TrafficBots, a multi-agent policy built upon motion prediction and end-to-end driving. Experiments on the open motion dataset show TrafficBots can simulate realistic multi-agent behaviors.
  arXiv  Detail & Related papers  (2023-03-07T18:28:41Z)
• Class-Aware Attention for Multimodal Trajectory Prediction [0.7130302992490973]
  We present a novel framework for multimodal trajectory prediction in autonomous driving. Our model is able to run in real-time, achieving a high inference rate of over 300 FPS.
  arXiv  Detail & Related papers  (2022-08-31T18:43:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.