An Expert Ensemble for Detecting Anomalous Scenes, Interactions, and Behaviors in Autonomous Driving

• URL: http://arxiv.org/abs/2502.16389v1
• Date: Sun, 23 Feb 2025 00:43:23 GMT
• Title: An Expert Ensemble for Detecting Anomalous Scenes, Interactions, and Behaviors in Autonomous Driving
• Authors: Tianchen Ji, Neeloy Chakraborty, Andre Schreiber, Katherine Driggs-Campbell,
• Abstract summary: Safety in a near-infinite number of driving scenarios becomes one of the major concerns for the widespread adoption of fully autonomous driving.<n>On-road anomaly detection in egocentric videos remains a challenging problem due to the difficulties introduced by complex and interactive scenarios.<n>We propose three unsupervised anomaly detection experts: a scene expert that focuses on frame-level appearances to detect abnormal scenes and unexpected scene motions; an interaction expert that models normal relative motions between two road participants and raises alarms whenever anomalous interactions emerge; and a behavior expert which monitors abnormal behaviors of individual objects by future trajectory prediction.
• Score: 5.424441980205082
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As automated vehicles enter public roads, safety in a near-infinite number of driving scenarios becomes one of the major concerns for the widespread adoption of fully autonomous driving. The ability to detect anomalous situations outside of the operational design domain is a key component in self-driving cars, enabling us to mitigate the impact of abnormal ego behaviors and to realize trustworthy driving systems. On-road anomaly detection in egocentric videos remains a challenging problem due to the difficulties introduced by complex and interactive scenarios. We conduct a holistic analysis of common on-road anomaly patterns, from which we propose three unsupervised anomaly detection experts: a scene expert that focuses on frame-level appearances to detect abnormal scenes and unexpected scene motions; an interaction expert that models normal relative motions between two road participants and raises alarms whenever anomalous interactions emerge; and a behavior expert which monitors abnormal behaviors of individual objects by future trajectory prediction. To combine the strengths of all the modules, we propose an expert ensemble (Xen) using a Kalman filter, in which the final anomaly score is absorbed as one of the states and the observations are generated by the experts. Our experiments employ a novel evaluation protocol for realistic model performance, demonstrate superior anomaly detection performance than previous methods, and show that our framework has potential in classifying anomaly types using unsupervised learning on a large-scale on-road anomaly dataset.

Related papers

• A Survey of World Models for Autonomous Driving [63.33363128964687]
  Recent breakthroughs in autonomous driving have been propelled by advances in robust world modeling.<n>This paper systematically reviews recent advances in world models for autonomous driving.
  arXiv  Detail & Related papers  (2025-01-20T04:00:02Z)
• GeneralAD: Anomaly Detection Across Domains by Attending to Distorted Features [68.14842693208465]
  GeneralAD is an anomaly detection framework designed to operate in semantic, near-distribution, and industrial settings. We propose a novel self-supervised anomaly generation module that employs straightforward operations like noise addition and shuffling to patch features. We extensively evaluated our approach on ten datasets, achieving state-of-the-art results in six and on-par performance in the remaining.
  arXiv  Detail & Related papers  (2024-07-17T09:27:41Z)
• Open-Vocabulary Video Anomaly Detection [57.552523669351636]
  Video anomaly detection (VAD) with weak supervision has achieved remarkable performance in utilizing video-level labels to discriminate whether a video frame is normal or abnormal. Recent studies attempt to tackle a more realistic setting, open-set VAD, which aims to detect unseen anomalies given seen anomalies and normal videos. This paper takes a step further and explores open-vocabulary video anomaly detection (OVVAD), in which we aim to leverage pre-trained large models to detect and categorize seen and unseen anomalies.
  arXiv  Detail & Related papers  (2023-11-13T02:54:17Z)
• Exploring the Potential of World Models for Anomaly Detection in Autonomous Driving [11.091582432763738]
  We show how world models can be leveraged to perform anomaly detection in the domain of autonomous driving. We provide a characterization of world models and relate individual components to previous works in anomaly detection.
  arXiv  Detail & Related papers  (2023-08-10T17:04:51Z)
• ReasonNet: End-to-End Driving with Temporal and Global Reasoning [31.319673950804972]
  We present ReasonNet, a novel end-to-end driving framework that extensively exploits both temporal and global information of the driving scene. Our method can effectively process the interactions and relationships among features in different frames. Reasoning about the global information of the scene can also improve overall perception performance.
  arXiv  Detail & Related papers  (2023-05-17T18:24:43Z)
• Learning Representation for Anomaly Detection of Vehicle Trajectories [15.20257956793474]
  Predicting the future trajectories of surrounding vehicles based on their history trajectories is a critical task in autonomous driving. Small crafted perturbations can significantly mislead the future trajectory prediction module of the ego vehicle. We propose two novel methods for learning effective and efficient representations for online anomaly detection of vehicle trajectories.
  arXiv  Detail & Related papers  (2023-03-09T02:48:59Z)
• An Outlier Exposure Approach to Improve Visual Anomaly Detection Performance for Mobile Robots [76.36017224414523]
  We consider the problem of building visual anomaly detection systems for mobile robots. Standard anomaly detection models are trained using large datasets composed only of non-anomalous data. We tackle the problem of exploiting these data to improve the performance of a Real-NVP anomaly detection model.
  arXiv  Detail & Related papers  (2022-09-20T15:18:13Z)
• Causality-Based Multivariate Time Series Anomaly Detection [63.799474860969156]
  We formulate the anomaly detection problem from a causal perspective and view anomalies as instances that do not follow the regular causal mechanism to generate the multivariate data. We then propose a causality-based anomaly detection approach, which first learns the causal structure from data and then infers whether an instance is an anomaly relative to the local causal mechanism. We evaluate our approach with both simulated and public datasets as well as a case study on real-world AIOps applications.
  arXiv  Detail & Related papers  (2022-06-30T06:00:13Z)
• Anomaly Detection in Multi-Agent Trajectories for Automated Driving [2.5211566369910967]
  Similar to humans, automated vehicles are supposed to perform anomaly detection. Our innovation is the ability to jointly learn multiple trajectories of a dynamic number of agents.
  arXiv  Detail & Related papers  (2021-10-15T08:07:31Z)
• Multi-Agent Variational Occlusion Inference Using People as Sensors [28.831182328958477]
  Inferring occupancy from agent behaviors is an inherently multimodal problem. We propose an occlusion inference method that characterizes observed behaviors of human agents as sensor measurements. Our approach is validated on a real-world dataset, outperforming baselines and demonstrating real-time capable performance.
  arXiv  Detail & Related papers  (2021-09-05T21:56:54Z)
• Anomaly Detection in Video via Self-Supervised and Multi-Task Learning [113.81927544121625]
  Anomaly detection in video is a challenging computer vision problem. In this paper, we approach anomalous event detection in video through self-supervised and multi-task learning at the object level.
  arXiv  Detail & Related papers  (2020-11-15T10:21:28Z)
• Studying Person-Specific Pointing and Gaze Behavior for Multimodal Referencing of Outside Objects from a Moving Vehicle [58.720142291102135]
  Hand pointing and eye gaze have been extensively investigated in automotive applications for object selection and referencing. Existing outside-the-vehicle referencing methods focus on a static situation, whereas the situation in a moving vehicle is highly dynamic and subject to safety-critical constraints. We investigate the specific characteristics of each modality and the interaction between them when used in the task of referencing outside objects.
  arXiv  Detail & Related papers  (2020-09-23T14:56:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.