Detecting Out-of-Distribution Objects through Class-Conditioned Inpainting

• URL: http://arxiv.org/abs/2402.03292v3
• Date: Mon, 09 Jun 2025 14:06:04 GMT
• Title: Detecting Out-of-Distribution Objects through Class-Conditioned Inpainting
• Authors: Quang-Huy Nguyen, Jin Peng Zhou, Zhenzhen Liu, Khanh-Huyen Bui, Kilian Q. Weinberger, Wei-Lun Chao, Dung D. Le,
• Abstract summary: Real-world deployment often introduces novel and unexpected objects, referred to as out-of-distribution (OOD) objects.<n>Modern object detectors are typically overconfident, making it unreliable to use their predictions alone for OOD detection.<n>We utilize an off-the-shelf text-to-image generative model, such as Stable Diffusion, which is trained with objective functions distinct from those of discriminative object detectors.
• Score: 42.67743584239442
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent object detectors have achieved impressive accuracy in identifying objects seen during training. However, real-world deployment often introduces novel and unexpected objects, referred to as out-of-distribution (OOD) objects, posing significant challenges to model trustworthiness. Modern object detectors are typically overconfident, making it unreliable to use their predictions alone for OOD detection. To address this, we propose leveraging an auxiliary model as a complementary solution. Specifically, we utilize an off-the-shelf text-to-image generative model, such as Stable Diffusion, which is trained with objective functions distinct from those of discriminative object detectors. We hypothesize that this fundamental difference enables the detection of OOD objects by measuring inconsistencies between the models. Concretely, for a given detected object bounding box and its predicted in-distribution class label, we perform class-conditioned inpainting on the image with the object removed. If the object is OOD, the inpainted image is likely to deviate significantly from the original, making the reconstruction error a robust indicator of OOD status. Extensive experiments demonstrate that our approach consistently surpasses existing zero-shot and non-zero-shot OOD detection methods, establishing a robust framework for enhancing object detection systems in dynamic environments.

Related papers

• Dream-Box: Object-wise Outlier Generation for Out-of-Distribution Detection [15.806236012151968]
  Out-of-distribution (OOD) detection is a challenging task that has received significant attention in recent years.<n>Recent work has focused on generating synthetic outliers and using them to train an outlier detector.<n>We introduce Dream-Box, a method that provides a link to object-wise outlier generation in the pixel space for OOD detection.
  arXiv  Detail & Related papers  (2025-04-25T23:52:27Z)
• VisTa: Visual-contextual and Text-augmented Zero-shot Object-level OOD Detection [22.200900846112805]
  We introduce a new method to adapt CLIP for zero-shot object-level OOD detection.<n>Our method preserves critical contextual information and improves the ability to differentiate between ID and OOD objects.
  arXiv  Detail & Related papers  (2025-03-28T10:08:17Z)
• RUNA: Object-level Out-of-Distribution Detection via Regional Uncertainty Alignment of Multimodal Representations [33.971901643313856]
  RUNA is a novel framework for detecting out-of-distribution (OOD) objects. It employs a regional uncertainty alignment mechanism to distinguish ID from OOD objects effectively. Our experiments show that RUNA substantially surpasses state-of-the-art methods in object-level OOD detection.
  arXiv  Detail & Related papers  (2025-03-28T10:01:55Z)
• Going Beyond Conventional OOD Detection [0.0]
  Out-of-distribution (OOD) detection is critical to ensure the safe deployment of deep learning models in critical applications. We present a unified Approach to Spurimatious, fine-grained, and Conventional OOD Detection (ASCOOD) Our approach effectively mitigates the impact of spurious correlations and encourages capturing fine-grained attributes.
  arXiv  Detail & Related papers  (2024-11-16T13:04:52Z)
• What If the Input is Expanded in OOD Detection? [77.37433624869857]
  Out-of-distribution (OOD) detection aims to identify OOD inputs from unknown classes. Various scoring functions are proposed to distinguish it from in-distribution (ID) data. We introduce a novel perspective, i.e., employing different common corruptions on the input space.
  arXiv  Detail & Related papers  (2024-10-24T06:47:28Z)
• Envisioning Outlier Exposure by Large Language Models for Out-of-Distribution Detection [71.93411099797308]
  Out-of-distribution (OOD) samples are crucial when deploying machine learning models in open-world scenarios. We propose to tackle this constraint by leveraging the expert knowledge and reasoning capability of large language models (LLM) to potential Outlier Exposure, termed EOE. EOE can be generalized to different tasks, including far, near, and fine-language OOD detection. EOE achieves state-of-the-art performance across different OOD tasks and can be effectively scaled to the ImageNet-1K dataset.
  arXiv  Detail & Related papers  (2024-06-02T17:09:48Z)
• Revisiting Out-of-Distribution Detection in LiDAR-based 3D Object Detection [12.633311483061647]
  Out-of-distribution (OOD) objects can lead to misclassifications, posing a significant risk to the safety and reliability of automated vehicles. We propose a new evaluation protocol that allows the use of existing datasets without modifying the point cloud. The effectiveness of our method is validated through experiments on the newly proposed nuScenes OOD benchmark.
  arXiv  Detail & Related papers  (2024-04-24T13:48:38Z)
• A noisy elephant in the room: Is your out-of-distribution detector robust to label noise? [49.88894124047644]
  We take a closer look at 20 state-of-the-art OOD detection methods. We show that poor separation between incorrectly classified ID samples vs. OOD samples is an overlooked yet important limitation of existing methods.
  arXiv  Detail & Related papers  (2024-04-02T09:40:22Z)
• From Global to Local: Multi-scale Out-of-distribution Detection [129.37607313927458]
  Out-of-distribution (OOD) detection aims to detect "unknown" data whose labels have not been seen during the in-distribution (ID) training process. Recent progress in representation learning gives rise to distance-based OOD detection. We propose Multi-scale OOD DEtection (MODE), a first framework leveraging both global visual information and local region details.
  arXiv  Detail & Related papers  (2023-08-20T11:56:25Z)
• Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection Capability [70.72426887518517]
  Out-of-distribution (OOD) detection is an indispensable aspect of secure AI when deploying machine learning models in real-world applications. We propose a novel method, Unleashing Mask, which aims to restore the OOD discriminative capabilities of the well-trained model with ID data. Our method utilizes a mask to figure out the memorized atypical samples, and then finetune the model or prune it with the introduced mask to forget them.
  arXiv  Detail & Related papers  (2023-06-06T14:23:34Z)
• Building One-class Detector for Anything: Open-vocabulary Zero-shot OOD Detection Using Text-image Models [23.302018871162186]
  We propose a novel one-class open-set OOD detector that leverages text-image pre-trained models in a zero-shot fashion. Our approach is designed to detect anything not in-domain and offers the flexibility to detect a wide variety of OOD. Our method shows superior performance over previous methods on all benchmarks.
  arXiv  Detail & Related papers  (2023-05-26T18:58:56Z)
• SOOD: Towards Semi-Supervised Oriented Object Detection [57.05141794402972]
  This paper proposes a novel Semi-supervised Oriented Object Detection model, termed SOOD, built upon the mainstream pseudo-labeling framework. Our experiments show that when trained with the two proposed losses, SOOD surpasses the state-of-the-art SSOD methods under various settings on the DOTA-v1.5 benchmark.
  arXiv  Detail & Related papers  (2023-04-10T11:10:42Z)
• Identifying Out-of-Distribution Samples in Real-Time for Safety-Critical 2D Object Detection with Margin Entropy Loss [0.0]
  We present an approach to enable OOD detection for 2D object detection by employing the margin entropy (ME) loss. A CNN trained with the ME loss significantly outperforms OOD detection using standard confidence scores.
  arXiv  Detail & Related papers  (2022-09-01T11:14:57Z)
• Adversarially-Aware Robust Object Detector [85.10894272034135]
  We propose a Robust Detector (RobustDet) based on adversarially-aware convolution to disentangle gradients for model learning on clean and adversarial images. Our model effectively disentangles gradients and significantly enhances the detection robustness with maintaining the detection ability on clean images.
  arXiv  Detail & Related papers  (2022-07-13T13:59:59Z)
• No Shifted Augmentations (NSA): compact distributions for robust self-supervised Anomaly Detection [4.243926243206826]
  Unsupervised Anomaly detection (AD) requires building a notion of normalcy, distinguishing in-distribution (ID) and out-of-distribution (OOD) data. We investigate how the emph geometrical compactness of the ID feature distribution makes isolating and detecting outliers easier. We propose novel architectural modifications to the self-supervised feature learning step, that enable such compact distributions for ID data to be learned.
  arXiv  Detail & Related papers  (2022-03-19T15:55:32Z)
• Suspected Object Matters: Rethinking Model's Prediction for One-stage Visual Grounding [93.82542533426766]
  We propose a Suspected Object Transformation mechanism (SOT) to encourage the target object selection among the suspected ones. SOT can be seamlessly integrated into existing CNN and Transformer-based one-stage visual grounders. Extensive experiments demonstrate the effectiveness of our proposed method.
  arXiv  Detail & Related papers  (2022-03-10T06:41:07Z)
• Exploiting Multi-Object Relationships for Detecting Adversarial Attacks in Complex Scenes [51.65308857232767]
  Vision systems that deploy Deep Neural Networks (DNNs) are known to be vulnerable to adversarial examples. Recent research has shown that checking the intrinsic consistencies in the input data is a promising way to detect adversarial attacks. We develop a novel approach to perform context consistency checks using language models.
  arXiv  Detail & Related papers  (2021-08-19T00:52:10Z)
• Triggering Failures: Out-Of-Distribution detection by learning from local adversarial attacks in Semantic Segmentation [76.2621758731288]
  We tackle the detection of out-of-distribution (OOD) objects in semantic segmentation. Our main contribution is a new OOD detection architecture called ObsNet associated with a dedicated training scheme based on Local Adversarial Attacks (LAA) We show it obtains top performances both in speed and accuracy when compared to ten recent methods of the literature on three different datasets.
  arXiv  Detail & Related papers  (2021-08-03T17:09:56Z)
• OODformer: Out-Of-Distribution Detection Transformer [15.17006322500865]
  In real-world safety-critical applications, it is important to be aware if a new data point is OOD. This paper proposes a first-of-its-kind OOD detection architecture named OODformer.
  arXiv  Detail & Related papers  (2021-07-19T15:46:38Z)
• Slender Object Detection: Diagnoses and Improvements [74.40792217534]
  In this paper, we are concerned with the detection of a particular type of objects with extreme aspect ratios, namely textbfslender objects. For a classical object detection method, a drastic drop of $18.9%$ mAP on COCO is observed, if solely evaluated on slender objects.
  arXiv  Detail & Related papers  (2020-11-17T09:39:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.