Run-time Monitoring of 3D Object Detection in Automated Driving Systems Using Early Layer Neural Activation Patterns

• URL: http://arxiv.org/abs/2404.07685v1
• Date: Thu, 11 Apr 2024 12:24:47 GMT
• Title: Run-time Monitoring of 3D Object Detection in Automated Driving Systems Using Early Layer Neural Activation Patterns
• Authors: Hakan Yekta Yatbaz, Mehrdad Dianati, Konstantinos Koufos, Roger Woodman,
• Abstract summary: Integrity monitoring of automated driving systems (ADS) is paramount for ensuring safety. Recent advancements in deep neural network (DNN)-based object detectors, their susceptibility to detection errors remains a significant concern.
• Score: 12.384452095533396
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Monitoring the integrity of object detection for errors within the perception module of automated driving systems (ADS) is paramount for ensuring safety. Despite recent advancements in deep neural network (DNN)-based object detectors, their susceptibility to detection errors, particularly in the less-explored realm of 3D object detection, remains a significant concern. State-of-the-art integrity monitoring (also known as introspection) mechanisms in 2D object detection mainly utilise the activation patterns in the final layer of the DNN-based detector's backbone. However, that may not sufficiently address the complexities and sparsity of data in 3D object detection. To this end, we conduct, in this article, an extensive investigation into the effects of activation patterns extracted from various layers of the backbone network for introspecting the operation of 3D object detectors. Through a comparative analysis using Kitti and NuScenes datasets with PointPillars and CenterPoint detectors, we demonstrate that using earlier layers' activation patterns enhances the error detection performance of the integrity monitoring system, yet increases computational complexity. To address the real-time operation requirements in ADS, we also introduce a novel introspection method that combines activation patterns from multiple layers of the detector's backbone and report its performance.

Related papers

• Uncertainty Estimation for 3D Object Detection via Evidential Learning [63.61283174146648]
  We introduce a framework for quantifying uncertainty in 3D object detection by leveraging an evidential learning loss on Bird's Eye View representations in the 3D detector. We demonstrate both the efficacy and importance of these uncertainty estimates on identifying out-of-distribution scenes, poorly localized objects, and missing (false negative) detections.
  arXiv  Detail & Related papers  (2024-10-31T13:13:32Z)
• Open-Set 3D object detection in LiDAR data as an Out-of-Distribution problem [6.131026007721572]
  3D Object Detection from LiDAR data has achieved industry-ready performance in controlled environments. Our work redefines the open-set 3D Object Detection problem in LiDAR data as an Out-Of-Distribution (OOD) problem to detect outlier objects.
  arXiv  Detail & Related papers  (2024-10-31T09:29:55Z)
• Integrity Monitoring of 3D Object Detection in Automated Driving Systems using Raw Activation Patterns and Spatial Filtering [12.384452095533396]
  The deep neural network (DNN) models are widely used for object detection in automated driving systems (ADS) Yet, such models are prone to errors which can have serious safety implications. Introspection and self-assessment models that aim to detect such errors are therefore of paramount importance for the safe deployment of ADS.
  arXiv  Detail & Related papers  (2024-05-13T10:03:03Z)
• Cross-Cluster Shifting for Efficient and Effective 3D Object Detection in Autonomous Driving [69.20604395205248]
  We present a new 3D point-based detector model, named Shift-SSD, for precise 3D object detection in autonomous driving. We introduce an intriguing Cross-Cluster Shifting operation to unleash the representation capacity of the point-based detector. We conduct extensive experiments on the KITTI, runtime, and nuScenes datasets, and the results demonstrate the state-of-the-art performance of Shift-SSD.
  arXiv  Detail & Related papers  (2024-03-10T10:36:32Z)
• Run-time Introspection of 2D Object Detection in Automated Driving Systems Using Learning Representations [13.529124221397822]
  We introduce a novel introspection solution for 2D object detection based on Deep Neural Networks (DNNs) We implement several state-of-the-art (SOTA) introspection mechanisms for error detection in 2D object detection, using one-stage and two-stage object detectors evaluated on KITTI and BDD datasets. Our performance evaluation shows that the proposed introspection solution outperforms SOTA methods, achieving an absolute reduction in the missed error ratio of 9% to 17% in the BDD dataset.
  arXiv  Detail & Related papers  (2024-03-02T10:56:14Z)
• Joint object detection and re-identification for 3D obstacle multi-camera systems [47.87501281561605]
  This research paper introduces a novel modification to an object detection network that uses camera and lidar information. It incorporates an additional branch designed for the task of re-identifying objects across adjacent cameras within the same vehicle. The results underscore the superiority of this method over traditional Non-Maximum Suppression (NMS) techniques.
  arXiv  Detail & Related papers  (2023-10-09T15:16:35Z)
• DetZero: Rethinking Offboard 3D Object Detection with Long-term Sequential Point Clouds [55.755450273390004]
  Existing offboard 3D detectors always follow a modular pipeline design to take advantage of unlimited sequential point clouds. We have found that the full potential of offboard 3D detectors is not explored mainly due to two reasons: (1) the onboard multi-object tracker cannot generate sufficient complete object trajectories, and (2) the motion state of objects poses an inevitable challenge for the object-centric refining stage. To tackle these problems, we propose a novel paradigm of offboard 3D object detection, named DetZero.
  arXiv  Detail & Related papers  (2023-06-09T16:42:00Z)
• AGO-Net: Association-Guided 3D Point Cloud Object Detection Network [86.10213302724085]
  We propose a novel 3D detection framework that associates intact features for objects via domain adaptation. We achieve new state-of-the-art performance on the KITTI 3D detection benchmark in both accuracy and speed.
  arXiv  Detail & Related papers  (2022-08-24T16:54:38Z)
• ST3D: Self-training for Unsupervised Domain Adaptation on 3D ObjectDetection [78.71826145162092]
  We present a new domain adaptive self-training pipeline, named ST3D, for unsupervised domain adaptation on 3D object detection from point clouds. Our ST3D achieves state-of-the-art performance on all evaluated datasets and even surpasses fully supervised results on KITTI 3D object detection benchmark.
  arXiv  Detail & Related papers  (2021-03-09T10:51:24Z)
• Online Monitoring of Object Detection Performance During Deployment [6.166295570030645]
  We introduce a cascaded neural network that monitors the performance of the object detector by predicting the quality of its mean average precision (mAP) on a sliding window of the input frames. We evaluate our proposed approach using different combinations of autonomous driving datasets and object detectors.
  arXiv  Detail & Related papers  (2020-11-16T07:01:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.