Calibrating Probabilistic Object Detectors with Annotator Disagreement

• URL: http://arxiv.org/abs/2605.24722v1
• Date: Sat, 23 May 2026 20:27:10 GMT
• Title: Calibrating Probabilistic Object Detectors with Annotator Disagreement
• Abstract summary: We introduce an efficient yet effective framework to calibrate probabilistic object detectors.<n>This framework is generalizable to many existing probabilistic object detectors.<n> Empirical results with real-world and synthetic datasets of medical and natural images demonstrate the superior performance of the proposed framework.
• Score: 2.6024316482367387
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: High degrees of disagreement among annotators can exist for ambiguous objects, e.g. in medical images, underscoring the challenges of establishing ground truth annotations in object detection tasks. Despite this, all existing object detectors implicitly require access to ground truth annotations for either training or evaluation. The fundamental questions we target are: How can we learn an object detector with multiple annotators' annotations but without objective ground truth annotations due to object ambiguity, and how can we enable the learned detector to express meaningful model predictive uncertainties in detecting ambiguous objects? To answer these questions, we present an interpretable approach to calibrate probabilistic object detectors, where the calibration goal is to align the class confidence and bounding box variance estimates to the annotators' annotation distribution. We introduce an efficient yet effective framework to calibrate probabilistic object detectors by designing four evaluation metrics to measure calibration errors regarding classification and localization, and proposing a train-time calibration and post-hoc calibrator, all without the need to access any ground truth. This framework is generalizable to many existing probabilistic object detectors, such as the YOLO families and two-stage detectors. Empirical results with real-world and synthetic datasets of medical and natural images demonstrate the superior performance of the proposed framework with three popular object detectors.

Related papers

• Uncertainty Estimation for 3D Object Detection via Evidential Learning [63.61283174146648]
  We introduce a framework for quantifying uncertainty in 3D object detection by leveraging an evidential learning loss on Bird's Eye View representations in the 3D detector. We demonstrate both the efficacy and importance of these uncertainty estimates on identifying out-of-distribution scenes, poorly localized objects, and missing (false negative) detections.
  arXiv  Detail & Related papers  (2024-10-31T13:13:32Z)
• Towards Building Self-Aware Object Detectors via Reliable Uncertainty Quantification and Calibration [17.461451218469062]
  In this work, we introduce the Self-Aware Object Detection (SAOD) task. The SAOD task respects and adheres to the challenges that object detectors face in safety-critical environments such as autonomous driving. We extensively use our framework, which introduces novel metrics and large scale test datasets, to test numerous object detectors.
  arXiv  Detail & Related papers  (2023-07-03T11:16:39Z)
• A Review of Uncertainty Calibration in Pretrained Object Detectors [5.440028715314566]
  We investigate the uncertainty calibration properties of different pretrained object detection architectures in a multi-class setting. We propose a framework to ensure a fair, unbiased, and repeatable evaluation. We deliver novel insights into why poor detector calibration emerges.
  arXiv  Detail & Related papers  (2022-10-06T14:06:36Z)
• GLENet: Boosting 3D Object Detectors with Generative Label Uncertainty Estimation [70.75100533512021]
  In this paper, we formulate the label uncertainty problem as the diversity of potentially plausible bounding boxes of objects. We propose GLENet, a generative framework adapted from conditional variational autoencoders, to model the one-to-many relationship between a typical 3D object and its potential ground-truth bounding boxes with latent variables. The label uncertainty generated by GLENet is a plug-and-play module and can be conveniently integrated into existing deep 3D detectors.
  arXiv  Detail & Related papers  (2022-07-06T06:26:17Z)
• Object Detection as Probabilistic Set Prediction [3.7599363231894176]
  We present a proper scoring rule for evaluating and training probabilistic object detectors. Our results indicate that the training of existing detectors is optimized toward non-probabilistic metrics.
  arXiv  Detail & Related papers  (2022-03-15T15:13:52Z)
• Contrastive Object Detection Using Knowledge Graph Embeddings [72.17159795485915]
  We compare the error statistics of the class embeddings learned from a one-hot approach with semantically structured embeddings from natural language processing or knowledge graphs. We propose a knowledge-embedded design for keypoint-based and transformer-based object detection architectures.
  arXiv  Detail & Related papers  (2021-12-21T17:10:21Z)
• Labels Are Not Perfect: Inferring Spatial Uncertainty in Object Detection [26.008419879970365]
  In this work, we infer the uncertainty in bounding box labels from LiDAR point clouds based on a generative model. Comprehensive experiments show that the proposed model reflects complex environmental noises in LiDAR perception and the label quality. We propose Jaccard IoU as a new evaluation metric that extends IoU by incorporating label uncertainty.
  arXiv  Detail & Related papers  (2020-12-18T09:11:44Z)
• Class-agnostic Object Detection [16.97782147401037]
  We propose class-agnostic object detection as a new problem that focuses on detecting objects irrespective of their object-classes. Specifically, the goal is to predict bounding boxes for all objects in an image but not their object-classes. We propose training and evaluation protocols for benchmarking class-agnostic detectors to advance future research in this domain.
  arXiv  Detail & Related papers  (2020-11-28T19:22:38Z)
• Slender Object Detection: Diagnoses and Improvements [74.40792217534]
  In this paper, we are concerned with the detection of a particular type of objects with extreme aspect ratios, namely textbfslender objects. For a classical object detection method, a drastic drop of $18.9%$ mAP on COCO is observed, if solely evaluated on slender objects.
  arXiv  Detail & Related papers  (2020-11-17T09:39:42Z)
• Black-box Explanation of Object Detectors via Saliency Maps [66.745167677293]
  We propose D-RISE, a method for generating visual explanations for the predictions of object detectors. We show that D-RISE can be easily applied to different object detectors including one-stage detectors such as YOLOv3 and two-stage detectors such as Faster-RCNN.
  arXiv  Detail & Related papers  (2020-06-05T02:13:35Z)
• Adaptive Object Detection with Dual Multi-Label Prediction [78.69064917947624]
  We propose a novel end-to-end unsupervised deep domain adaptation model for adaptive object detection. The model exploits multi-label prediction to reveal the object category information in each image. We introduce a prediction consistency regularization mechanism to assist object detection.
  arXiv  Detail & Related papers  (2020-03-29T04:23:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.