Cost-Sensitive Uncertainty-Based Failure Recognition for Object Detection

• URL: http://arxiv.org/abs/2404.17427v1
• Date: Fri, 26 Apr 2024 14:03:55 GMT
• Title: Cost-Sensitive Uncertainty-Based Failure Recognition for Object Detection
• Authors: Moussa Kassem Sbeyti, Michelle Karg, Christian Wirth, Nadja Klein, Sahin Albayrak,
• Abstract summary: We propose a cost-sensitive framework for object detection tailored to user-defined budgets. We derive minimum thresholding requirements to prevent performance degradation. We automate and optimize the thresholding process to maximize the failure recognition rate.
• Score: 1.8990839669542954
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Object detectors in real-world applications often fail to detect objects due to varying factors such as weather conditions and noisy input. Therefore, a process that mitigates false detections is crucial for both safety and accuracy. While uncertainty-based thresholding shows promise, previous works demonstrate an imperfect correlation between uncertainty and detection errors. This hinders ideal thresholding, prompting us to further investigate the correlation and associated cost with different types of uncertainty. We therefore propose a cost-sensitive framework for object detection tailored to user-defined budgets on the two types of errors, missing and false detections. We derive minimum thresholding requirements to prevent performance degradation and define metrics to assess the applicability of uncertainty for failure recognition. Furthermore, we automate and optimize the thresholding process to maximize the failure recognition rate w.r.t. the specified budget. Evaluation on three autonomous driving datasets demonstrates that our approach significantly enhances safety, particularly in challenging scenarios. Leveraging localization aleatoric uncertainty and softmax-based entropy only, our method boosts the failure recognition rate by 36-60\% compared to conventional approaches. Code is available at https://mos-ks.github.io/publications.

Related papers

• Know Where You're Uncertain When Planning with Multimodal Foundation Models: A Formal Framework [54.40508478482667]
  We present a comprehensive framework to disentangle, quantify, and mitigate uncertainty in perception and plan generation. We propose methods tailored to the unique properties of perception and decision-making. We show that our uncertainty disentanglement framework reduces variability by up to 40% and enhances task success rates by 5% compared to baselines.
  arXiv  Detail & Related papers  (2024-11-03T17:32:00Z)
• Uncertainty Estimation for 3D Object Detection via Evidential Learning [63.61283174146648]
  We introduce a framework for quantifying uncertainty in 3D object detection by leveraging an evidential learning loss on Bird's Eye View representations in the 3D detector. We demonstrate both the efficacy and importance of these uncertainty estimates on identifying out-of-distribution scenes, poorly localized objects, and missing (false negative) detections.
  arXiv  Detail & Related papers  (2024-10-31T13:13:32Z)
• Learning a Factorized Orthogonal Latent Space using Encoder-only Architecture for Fault Detection; An Alarm management perspective [0.2455468619225742]
  This paper introduces a novel encoder-based residual design that effectively decouples erroneously identified and deterministic components of process variables. The proposed model employs two distinct encoders to factorize the latent space into two spaces: one for the deterministic part and the other for the part. The proposed model significantly enhances prediction quality while achieving nearly zero false alarms and missed detections.
  arXiv  Detail & Related papers  (2024-08-24T09:00:45Z)
• Predicting Safety Misbehaviours in Autonomous Driving Systems using Uncertainty Quantification [8.213390074932132]
  This paper evaluates different uncertainty quantification methods from the deep learning domain for the anticipatory testing of safety-critical misbehaviours. We compute uncertainty scores as the vehicle executes, following the intuition that high uncertainty scores are indicative of unsupported runtime conditions. In our study, we conducted an evaluation of the effectiveness and computational overhead associated with two uncertainty quantification methods, namely MC- Dropout and Deep Ensembles, for misbehaviour avoidance.
  arXiv  Detail & Related papers  (2024-04-29T10:28:28Z)
• Unsupervised Anomaly Detection with Rejection [19.136286864839846]
  Anomaly detectors learn a decision boundary by employing intuitions, which are hard to verify in practice. A way to combat this is by allowing the detector to reject examples with high uncertainty. This requires employing a confidence metric that captures the distance to the decision boundary and setting a rejection threshold to reject low-confidence predictions.
  arXiv  Detail & Related papers  (2023-05-22T16:22:32Z)
• Bayesian autoencoders with uncertainty quantification: Towards trustworthy anomaly detection [78.24964622317634]
  In this work, the formulation of Bayesian autoencoders (BAEs) is adopted to quantify the total anomaly uncertainty. To evaluate the quality of uncertainty, we consider the task of classifying anomalies with the additional option of rejecting predictions of high uncertainty. Our experiments demonstrate the effectiveness of the BAE and total anomaly uncertainty on a set of benchmark datasets and two real datasets for manufacturing.
  arXiv  Detail & Related papers  (2022-02-25T12:20:04Z)
• CertainNet: Sampling-free Uncertainty Estimation for Object Detection [65.28989536741658]
  Estimating the uncertainty of a neural network plays a fundamental role in safety-critical settings. In this work, we propose a novel sampling-free uncertainty estimation method for object detection. We call it CertainNet, and it is the first to provide separate uncertainties for each output signal: objectness, class, location and size.
  arXiv  Detail & Related papers  (2021-10-04T17:59:31Z)
• Temporal Difference Uncertainties as a Signal for Exploration [76.6341354269013]
  An effective approach to exploration in reinforcement learning is to rely on an agent's uncertainty over the optimal policy. In this paper, we highlight that value estimates are easily biased and temporally inconsistent. We propose a novel method for estimating uncertainty over the value function that relies on inducing a distribution over temporal difference errors.
  arXiv  Detail & Related papers  (2020-10-05T18:11:22Z)
• Labels Are Not Perfect: Improving Probabilistic Object Detection via Label Uncertainty [12.531126969367774]
  We leverage our previously proposed method for estimating uncertainty inherent in ground truth bounding box parameters. Experimental results on the KITTI dataset show that our method surpasses both the baseline model and the models based on simple uncertaintys by up to 3.6% in terms of Average Precision.
  arXiv  Detail & Related papers  (2020-08-10T14:49:49Z)
• Localization Uncertainty Estimation for Anchor-Free Object Detection [48.931731695431374]
  There are several limitations of the existing uncertainty estimation methods for anchor-based object detection. We propose a new localization uncertainty estimation method called UAD for anchor-free object detection. Our method captures the uncertainty in four directions of box offsets that are homogeneous, so that it can tell which direction is uncertain.
  arXiv  Detail & Related papers  (2020-06-28T13:49:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.