Exploring the Potential of World Models for Anomaly Detection in Autonomous Driving

• URL: http://arxiv.org/abs/2308.05701v2
• Date: Mon, 18 Sep 2023 11:32:53 GMT
• Title: Exploring the Potential of World Models for Anomaly Detection in Autonomous Driving
• Authors: Daniel Bogdoll, Lukas Bosch, Tim Joseph, Helen Gremmelmaier, Yitian Yang, J. Marius Z\"ollner
• Abstract summary: We show how world models can be leveraged to perform anomaly detection in the domain of autonomous driving. We provide a characterization of world models and relate individual components to previous works in anomaly detection.
• Score: 11.091582432763738
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In recent years there have been remarkable advancements in autonomous driving. While autonomous vehicles demonstrate high performance in closed-set conditions, they encounter difficulties when confronted with unexpected situations. At the same time, world models emerged in the field of model-based reinforcement learning as a way to enable agents to predict the future depending on potential actions. This led to outstanding results in sparse reward and complex control tasks. This work provides an overview of how world models can be leveraged to perform anomaly detection in the domain of autonomous driving. We provide a characterization of world models and relate individual components to previous works in anomaly detection to facilitate further research in the field.

Related papers

• Characterized Diffusion Networks for Enhanced Autonomous Driving Trajectory Prediction [0.6202955567445396]
  We present a novel trajectory prediction model for autonomous driving. Our model enhances the accuracy and reliability of trajectory predictions by incorporating uncertainty estimation and complex agent interactions. The proposed model showcases strong potential for application in real-world autonomous driving systems.
  arXiv  Detail & Related papers  (2024-11-25T15:03:44Z)
• Exploring the Interplay Between Video Generation and World Models in Autonomous Driving: A Survey [61.39993881402787]
  World models and video generation are pivotal technologies in the domain of autonomous driving. This paper investigates the relationship between these two technologies. By analyzing the interplay between video generation and world models, this survey identifies critical challenges and future research directions.
  arXiv  Detail & Related papers  (2024-11-05T08:58:35Z)
• Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Models [60.87795376541144]
  A world model is a neural network capable of predicting an agent's next state given past states and actions. During end-to-end training, our policy learns how to recover from errors by aligning with states observed in human demonstrations. We present qualitative and quantitative results, demonstrating significant improvements upon prior state of the art in closed-loop testing.
  arXiv  Detail & Related papers  (2024-09-25T06:48:25Z)
• Learning Multiple Probabilistic Decisions from Latent World Model in Autonomous Driving [40.4491758280365]
  Autoregressive world model exhibits robust generalization capabilities but encounters difficulties in deriving actions due to insufficient uncertainty modeling and self-delusion. We propose LatentDriver, a framework models the environment's next states and the ego vehicle's possible actions as a mixture distribution. LatentDriver surpasses state-of-the-art reinforcement learning and imitation learning methods, achieving expert-level performance.
  arXiv  Detail & Related papers  (2024-09-24T04:26:24Z)
• AIDE: An Automatic Data Engine for Object Detection in Autonomous Driving [68.73885845181242]
  We propose an Automatic Data Engine (AIDE) that automatically identifies issues, efficiently curates data, improves the model through auto-labeling, and verifies the model through generation of diverse scenarios. We further establish a benchmark for open-world detection on AV datasets to comprehensively evaluate various learning paradigms, demonstrating our method's superior performance at a reduced cost.
  arXiv  Detail & Related papers  (2024-03-26T04:27:56Z)
• GenAD: Generalized Predictive Model for Autonomous Driving [75.39517472462089]
  We introduce the first large-scale video prediction model in the autonomous driving discipline. Our model, dubbed GenAD, handles the challenging dynamics in driving scenes with novel temporal reasoning blocks. It can be adapted into an action-conditioned prediction model or a motion planner, holding great potential for real-world driving applications.
  arXiv  Detail & Related papers  (2024-03-14T17:58:33Z)
• World Models for Autonomous Driving: An Initial Survey [16.448614804069674]
  The capability to accurately predict future events and assess their implications is paramount for both safety and efficiency. World models have emerged as a transformative approach, enabling autonomous driving systems to synthesize and interpret vast amounts of sensor data. This paper provides an initial review of the current state and prospective advancements of world models in autonomous driving.
  arXiv  Detail & Related papers  (2024-03-05T03:23:55Z)
• GAIA-1: A Generative World Model for Autonomous Driving [9.578453700755318]
  We introduce GAIA-1 ('Generative AI for Autonomy'), a generative world model that generates realistic driving scenarios. Emerging properties from our model include learning high-level structures and scene dynamics, contextual awareness, generalization, and understanding of geometry.
  arXiv  Detail & Related papers  (2023-09-29T09:20:37Z)
• Model-Based Reinforcement Learning with Isolated Imaginations [61.67183143982074]
  We propose Iso-Dream++, a model-based reinforcement learning approach. We perform policy optimization based on the decoupled latent imaginations. This enables long-horizon visuomotor control tasks to benefit from isolating mixed dynamics sources in the wild.
  arXiv  Detail & Related papers  (2023-03-27T02:55:56Z)
• Isolating and Leveraging Controllable and Noncontrollable Visual Dynamics in World Models [65.97707691164558]
  We present Iso-Dream, which improves the Dream-to-Control framework in two aspects. First, by optimizing inverse dynamics, we encourage world model to learn controllable and noncontrollable sources. Second, we optimize the behavior of the agent on the decoupled latent imaginations of the world model.
  arXiv  Detail & Related papers  (2022-05-27T08:07:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.