Methodology for a Statistical Analysis of Influencing Factors on 3D Object Detection Performance

• URL: http://arxiv.org/abs/2411.08482v2
• Date: Sat, 01 Feb 2025 02:05:35 GMT
• Title: Methodology for a Statistical Analysis of Influencing Factors on 3D Object Detection Performance
• Authors: Anton Kuznietsov, Dirk Schweickard, Steven Peters,
• Abstract summary: In automated driving, object detection is an essential task to perceive the environment by localizing and classifying objects.<n>Most object detection algorithms are based on deep learning for superior performance.<n>We propose a first-of-its-kind methodology for analyzing the influence of various factors related to the objects or the environment on the detection performance of both LiDAR- and camera-based 3D object detectors.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In automated driving, object detection is an essential task to perceive the environment by localizing and classifying objects. Most object detection algorithms are based on deep learning for superior performance. However, their black-box nature makes it challenging to ensure safety. In this paper, we propose a first-of-its-kind methodology for analyzing the influence of various factors related to the objects or the environment on the detection performance of both LiDAR- and camera-based 3D object detectors. We conduct a statistical univariate analysis between each factor and the detection error on pedestrians to compare their strength of influence. In addition to univariate analysis, we employ a Random Forest (RF) model to predict the errors of specific detectors based on the provided meta-information. To interpret the predictions of the RF and assess the importance of individual features, we compute Shapley Values. By considering feature dependencies, the RF captures more complex relationships between meta-information and detection errors, allowing a more nuanced analysis of the factors contributing to the observed errors. Recognizing the factors that influence detection performance helps identify performance insufficiencies in the trained object detector and supports the safe development of object detection systems.

Related papers

• Uncertainty Representation in a SOTIF-Related Use Case with Dempster-Shafer Theory for LiDAR Sensor-Based Object Detection [0.0]
  Uncertainty in LiDAR sensor-based object detection arises from environmental variability and sensor performance limitations. Dempster-Shafer Theory (DST) is employed to construct a Frame of Discernment (FoD) to represent detection outcomes. Yager's Rule of Combination is used to resolve conflicting evidence from multiple sources.
  arXiv  Detail & Related papers  (2025-03-03T22:13:51Z)
• A Hybrid Framework for Statistical Feature Selection and Image-Based Noise-Defect Detection [55.2480439325792]
  This paper presents a hybrid framework that integrates both statistical feature selection and classification techniques to improve defect detection accuracy. We present around 55 distinguished features that are extracted from industrial images, which are then analyzed using statistical methods. By integrating these methods with flexible machine learning applications, the proposed framework improves detection accuracy and reduces false positives and misclassifications.
  arXiv  Detail & Related papers  (2024-12-11T22:12:21Z)
• Uncertainty Estimation for 3D Object Detection via Evidential Learning [63.61283174146648]
  We introduce a framework for quantifying uncertainty in 3D object detection by leveraging an evidential learning loss on Bird's Eye View representations in the 3D detector. We demonstrate both the efficacy and importance of these uncertainty estimates on identifying out-of-distribution scenes, poorly localized objects, and missing (false negative) detections.
  arXiv  Detail & Related papers  (2024-10-31T13:13:32Z)
• Perceptual Piercing: Human Visual Cue-based Object Detection in Low Visibility Conditions [2.0409124291940826]
  This study proposes a novel deep learning framework inspired by atmospheric scattering and human visual cortex mechanisms to enhance object detection under poor visibility scenarios such as fog, smoke, and haze. The objective is to enhance the precision and reliability of detection systems under adverse environmental conditions.
  arXiv  Detail & Related papers  (2024-10-02T04:03:07Z)
• Fairness in Autonomous Driving: Towards Understanding Confounding Factors in Object Detection under Challenging Weather [7.736445799116692]
  This study provides an empirical analysis of fairness in detecting pedestrians in a state-of-the-art transformer-based object detector. In addition to classical metrics, we introduce novel probability-based metrics to measure various intricate properties of object detection. Our quantitative analysis reveals how the previously overlooked yet intuitive factors, such as the distribution of demographic groups in the scene, the severity of weather, the pedestrians' proximity to the AV, among others, affect object detection performance.
  arXiv  Detail & Related papers  (2024-05-31T22:35:10Z)
• Integrity Monitoring of 3D Object Detection in Automated Driving Systems using Raw Activation Patterns and Spatial Filtering [12.384452095533396]
  The deep neural network (DNN) models are widely used for object detection in automated driving systems (ADS) Yet, such models are prone to errors which can have serious safety implications. Introspection and self-assessment models that aim to detect such errors are therefore of paramount importance for the safe deployment of ADS.
  arXiv  Detail & Related papers  (2024-05-13T10:03:03Z)
• Object criticality for safer navigation [1.565361244756411]
  Given an object detector, filtering objects based on their relevance, reduces the risk of missing relevant objects, decreases the likelihood of dangerous trajectories, and improves the quality of trajectories in general. We show that, given an object detector, filtering objects based on their relevance, reduces the risk of missing relevant objects, decreases the likelihood of dangerous trajectories, and improves the quality of trajectories in general.
  arXiv  Detail & Related papers  (2024-04-25T09:02:22Z)
• Machine Learning for Pre/Post Flight UAV Rotor Defect Detection Using Vibration Analysis [54.550658461477106]
  Unmanned Aerial Vehicles (UAVs) will be critical infrastructural components of future smart cities. In order to operate efficiently, UAV reliability must be ensured by constant monitoring for faults and failures. This paper leverages signal processing and Machine Learning methods to analyze the data of a comprehensive vibrational analysis to determine the presence of rotor blade defects.
  arXiv  Detail & Related papers  (2024-04-24T13:50:27Z)
• Innovative Horizons in Aerial Imagery: LSKNet Meets DiffusionDet for Advanced Object Detection [55.2480439325792]
  We present an in-depth evaluation of an object detection model that integrates the LSKNet backbone with the DiffusionDet head. The proposed model achieves a mean average precision (MAP) of approximately 45.7%, which is a significant improvement. This advancement underscores the effectiveness of the proposed modifications and sets a new benchmark in aerial image analysis.
  arXiv  Detail & Related papers  (2023-11-21T19:49:13Z)
• DRUformer: Enhancing the driving scene Important object detection with driving relationship self-understanding [50.81809690183755]
  Traffic accidents frequently lead to fatal injuries, contributing to over 50 million deaths until 2023. Previous research primarily assessed the importance of individual participants, treating them as independent entities. We introduce Driving scene Relationship self-Understanding transformer (DRUformer) to enhance the important object detection task.
  arXiv  Detail & Related papers  (2023-11-11T07:26:47Z)
• The Impact of Different Backbone Architecture on Autonomous Vehicle Dataset [120.08736654413637]
  The quality of the features extracted by the backbone architecture can have a significant impact on the overall detection performance. Our study evaluates three well-known autonomous vehicle datasets, namely KITTI, NuScenes, and BDD, to compare the performance of different backbone architectures on object detection tasks.
  arXiv  Detail & Related papers  (2023-09-15T17:32:15Z)
• A Meta-level Analysis of Online Anomaly Detectors [4.852567314334134]
  Real-time detection of anomalies in streaming data is receiving increasing attention. Yet, little attention has been given to compare the effectiveness and efficiency of anomaly detectors for streaming data. We present a qualitative, synthetic overview of major online detectors from different algorithmic families.
  arXiv  Detail & Related papers  (2022-09-13T11:28:15Z)
• A Quality Index Metric and Method for Online Self-Assessment of Autonomous Vehicles Sensory Perception [164.93739293097605]
  We propose a novel evaluation metric, named as the detection quality index (DQI), which assesses the performance of camera-based object detection algorithms. We have developed a superpixel-based attention network (SPA-NET) that utilizes raw image pixels and superpixels as input to predict the proposed DQI evaluation metric.
  arXiv  Detail & Related papers  (2022-03-04T22:16:50Z)
• Evaluating Object (mis)Detection from a Safety and Reliability Perspective: Discussion and Measures [1.8492669447784602]
  We propose new object detection measures that reward the correct identification of objects that are most dangerous and most likely to affect driving decisions. We apply our model on the recent autonomous driving dataset nuScenes, and we compare nine object detectors. Results show that, in several settings, object detectors that perform best according to the nuScenes ranking are not the preferable ones when the focus is shifted on safety and reliability.
  arXiv  Detail & Related papers  (2022-03-04T09:31:20Z)
• Comprehensive Analysis of the Object Detection Pipeline on UAVs [16.071349046409885]
  We first empirically analyze the influence of seven parameters (quantization, compression, resolution, color model, image distortion, gamma correction, additional channels) in remote sensing applications. We show that not all parameters have an equal impact on detection accuracy and data throughput, and that by using a suitable compromise between parameters we are able to improve detection accuracy for lightweight object detection models.
  arXiv  Detail & Related papers  (2022-03-01T09:30:01Z)
• Comparative study of 3D object detection frameworks based on LiDAR data and sensor fusion techniques [0.0]
  The perception system plays a significant role in providing an accurate interpretation of a vehicle's environment in real-time. Deep learning techniques transform the huge amount of data from the sensors into semantic information. 3D object detection methods, by utilizing the additional pose data from the sensors such as LiDARs, stereo cameras, provides information on the size and location of the object.
  arXiv  Detail & Related papers  (2022-02-05T09:34:58Z)
• Robust and Accurate Object Detection via Adversarial Learning [111.36192453882195]
  This work augments the fine-tuning stage for object detectors by exploring adversarial examples. Our approach boosts the performance of state-of-the-art EfficientDets by +1.1 mAP on the object detection benchmark.
  arXiv  Detail & Related papers  (2021-03-23T19:45:26Z)
• Slender Object Detection: Diagnoses and Improvements [74.40792217534]
  In this paper, we are concerned with the detection of a particular type of objects with extreme aspect ratios, namely textbfslender objects. For a classical object detection method, a drastic drop of $18.9%$ mAP on COCO is observed, if solely evaluated on slender objects.
  arXiv  Detail & Related papers  (2020-11-17T09:39:42Z)
• Adaptive Object Detection with Dual Multi-Label Prediction [78.69064917947624]
  We propose a novel end-to-end unsupervised deep domain adaptation model for adaptive object detection. The model exploits multi-label prediction to reveal the object category information in each image. We introduce a prediction consistency regularization mechanism to assist object detection.
  arXiv  Detail & Related papers  (2020-03-29T04:23:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.