Know Your Space: Inlier and Outlier Construction for Calibrating Medical OOD Detectors

• URL: http://arxiv.org/abs/2207.05286v2
• Date: Sat, 22 Apr 2023 15:31:55 GMT
• Title: Know Your Space: Inlier and Outlier Construction for Calibrating Medical OOD Detectors
• Authors: Vivek Narayanaswamy, Yamen Mubarka, Rushil Anirudh, Deepta Rajan, Andreas Spanias and Jayaraman J. Thiagarajan
• Abstract summary: We focus on the problem of producing well-calibrated out-of-distribution (OOD) detectors. Motivated by the difficulty of curating suitable calibration datasets, synthetic augmentations have become highly prevalent for inlier/outlier specification. We find that the optimal protocol is to synthesize latent-space inliers along with diverse pixel-space outliers.
• Score: 39.8194799829348
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We focus on the problem of producing well-calibrated out-of-distribution (OOD) detectors, in order to enable safe deployment of medical image classifiers. Motivated by the difficulty of curating suitable calibration datasets, synthetic augmentations have become highly prevalent for inlier/outlier specification. While there have been rapid advances in data augmentation techniques, this paper makes a striking finding that the space in which the inliers and outliers are synthesized, in addition to the type of augmentation, plays a critical role in calibrating OOD detectors. Using the popular energy-based OOD detection framework, we find that the optimal protocol is to synthesize latent-space inliers along with diverse pixel-space outliers. Based on empirical studies with multiple medical imaging benchmarks, we demonstrate that our approach consistently leads to superior OOD detection ($15\% - 35\%$ in AUROC) over the state-of-the-art in a variety of open-set recognition settings.

Related papers

• Enhancing Out-of-Distribution Detection with Extended Logit Normalization [8.243349010573242]
  Out-of-distribution (OOD) detection is essential for the safe deployment of machine learning models. Recent advances have explored improved classification losses and representation learning strategies to enhance OOD detection. These methods are often tailored to specific post-hoc detection techniques, limiting their generalizability.
  arXiv  Detail & Related papers  (2025-04-15T17:51:35Z)
• Revisiting Medical Image Retrieval via Knowledge Consolidation [46.6989555659494]
  We propose a novel method to consolidate knowledge of hierarchical features and functions. We introduce Depth-aware Representation Fusion (DaRF) and Structure-aware Contrastive Hashing (SCH) Our method achieves a 5.6-38.9% improvement in mean Average Precision on the anatomical radiology dataset.
  arXiv  Detail & Related papers  (2025-03-12T13:16:42Z)
• Enhancing Reconstruction-Based Out-of-Distribution Detection in Brain MRI with Model and Metric Ensembles [2.6123133623254193]
  Out-of-distribution (OOD) detection is crucial for safely deploying automated medical image analysis systems. We investigated the effectiveness of a reconstruction-based autoencoder for unsupervised detection of synthetic artifacts in brain MRI.
  arXiv  Detail & Related papers  (2024-12-23T13:58:52Z)
• Margin-bounded Confidence Scores for Out-of-Distribution Detection [2.373572816573706]
  We propose a novel method called Margin bounded Confidence Scores (MaCS) to address the nontrivial OOD detection problem. MaCS enlarges the disparity between ID and OOD scores, which in turn makes the decision boundary more compact. Experiments on various benchmark datasets for image classification tasks demonstrate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2024-09-22T05:40:25Z)
• Dissecting Out-of-Distribution Detection and Open-Set Recognition: A Critical Analysis of Methods and Benchmarks [17.520137576423593]
  We aim to provide a consolidated view of the two largest sub-fields within the community: out-of-distribution (OOD) detection and open-set recognition (OSR) We perform rigorous cross-evaluation between state-of-the-art methods in the OOD detection and OSR settings and identify a strong correlation between the performances of methods for them. We propose a new, large-scale benchmark setting which we suggest better disentangles the problem tackled by OOD detection and OSR.
  arXiv  Detail & Related papers  (2024-08-29T17:55:07Z)
• Enhancing OOD Detection Using Latent Diffusion [5.093257685701887]
  Out-of-Distribution (OOD) detection algorithms have been developed to identify unknown samples or objects in real-world deployments. We propose an Outlier Aware Learning framework, which synthesizes OOD training data in the latent space. We also develop a knowledge distillation module to prevent the degradation of ID classification accuracy when training with OOD data.
  arXiv  Detail & Related papers  (2024-06-24T11:01:43Z)
• Automatic diagnosis of knee osteoarthritis severity using Swin transformer [55.01037422579516]
  Knee osteoarthritis (KOA) is a widespread condition that can cause chronic pain and stiffness in the knee joint. We propose an automated approach that employs the Swin Transformer to predict the severity of KOA.
  arXiv  Detail & Related papers  (2023-07-10T09:49:30Z)
• Rethinking Out-of-distribution (OOD) Detection: Masked Image Modeling is All You Need [52.88953913542445]
  We find surprisingly that simply using reconstruction-based methods could boost the performance of OOD detection significantly. We take Masked Image Modeling as a pretext task for our OOD detection framework (MOOD)
  arXiv  Detail & Related papers  (2023-02-06T08:24:41Z)
• OpenOOD: Benchmarking Generalized Out-of-Distribution Detection [60.13300701826931]
  Out-of-distribution (OOD) detection is vital to safety-critical machine learning applications. The field currently lacks a unified, strictly formulated, and comprehensive benchmark. We build a unified, well-structured called OpenOOD, which implements over 30 methods developed in relevant fields.
  arXiv  Detail & Related papers  (2022-10-13T17:59:57Z)
• DIAGNOSE: Avoiding Out-of-distribution Data using Submodular Information Measures [13.492292022589918]
  We propose Diagnose, a novel active learning framework that can jointly model similarity and dissimilarity. Our experiments verify the superiority of Diagnose over the state-of-the-art AL methods across multiple domains of medical imaging.
  arXiv  Detail & Related papers  (2022-10-04T11:07:48Z)
• How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection? [22.519572587827213]
  CIDER is a representation learning framework that exploits hyperspherical embeddings for OOD detection. CIDER establishes superior performance, outperforming the latest rival by 19.36% in FPR95.
  arXiv  Detail & Related papers  (2022-03-08T23:44:01Z)
• DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations [90.27736364704108]
  We present DrugOOD, a systematic OOD dataset curator and benchmark for AI-aided drug discovery. DrugOOD comes with an open-source Python package that fully automates benchmarking processes. We focus on one of the most crucial problems in AIDD: drug target binding affinity prediction.
  arXiv  Detail & Related papers  (2022-01-24T12:32:48Z)
• CVAD: A generic medical anomaly detector based on Cascade VAE [2.647674705784439]
  We focus on the generalizability of OOD detection for medical images and propose a self-supervised Cascade Variational autoencoder-based Anomaly Detector (CVAD) We use a variational autoencoders' cascade architecture, which combines latent representation at multiple scales, before being fed to a discriminator to distinguish the OOD data from the in-distribution (ID) data. We compare the performance with the state-of-the-art deep learning models to demonstrate our model's efficacy on various open-access medical imaging datasets for both intra- and inter-class OOD.
  arXiv  Detail & Related papers  (2021-10-29T14:20:43Z)
• Robust Out-of-distribution Detection for Neural Networks [51.19164318924997]
  We show that existing detection mechanisms can be extremely brittle when evaluating on in-distribution and OOD inputs. We propose an effective algorithm called ALOE, which performs robust training by exposing the model to both adversarially crafted inlier and outlier examples.
  arXiv  Detail & Related papers  (2020-03-21T17:46:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.