Layer Selection in VLMs for Zero-Shot OOD Detection via Multi-Resolution Entropy Estimation

• URL: http://arxiv.org/abs/2609.08524v1
• Date: Tue, 08 Sep 2026 10:12:52 GMT
• Title: Layer Selection in VLMs for Zero-Shot OOD Detection via Multi-Resolution Entropy Estimation
• Abstract summary: Out-of-distribution (OOD) detection is crucial for safe deployment of medical AI systems.<n>We show that existing methods rely almost exclusively on final-layer embeddings, implicitly assuming that the deepest representations are universally optimal.<n>We propose a multi-resolution entropy estimation strategy that aggregates histogram statistics across multiple discretization scales.
• Score: 6.633563728823828
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Out-of-distribution (OOD) detection is crucial for safe deployment of medical AI systems, where domain shifts arise across institutions, acquisition protocols, and patient populations. VLMs enable zero-shot OOD detection by embedding images into a language-aligned latent space, where cross-modal similarity serves as a non-parametric confidence signal for identifying in-distribution samples. Yet existing methods rely almost exclusively on final-layer embeddings, implicitly assuming that the deepest representations are universally optimal. We first show that this assumption does not hold in medical imaging: intermediate layers provide complementary OOD signals, and the optimal representational depth depends on the respective image modality. While prior work selects layer combinations via entropy minimization of normalized histograms, we demonstrate that single-resolution entropy estimation is highly sensitive to binning choices, leading to performance variations of up to 19.3% AUROC. To address this instability, we propose a multi-resolution entropy estimation strategy that aggregates histogram statistics across multiple discretization scales, enabling robust and stable intermediate-layer selection. Across two medical OOD benchmarks, namely MIDOG and OASIS, covering distinct imaging modalities, diverse shift types, and different VLM backbones, our method consistently outperforms state-of-the-art approaches, offering a lightweight and stable solution for zero-shot OOD detection.

Related papers

• Unified Multi-Layer Subspace Modeling for Cross-Domain OOD Detection [12.06155749281499]
  PRISM is a model-agnostic post-hoc OOD detection method.<n>It fuses intermediate and deep features into a single hierarchical embedding.<n>It captures both in-subspace semantic deviations and off-subspace anomalies.
  arXiv  Detail & Related papers  (2026-09-06T19:00:54Z)
• Bringing Multimodal Large Language Models to Infrared-Visible Image Fusion Quality Assessment [48.882448108261826]
  Infrared-Visible image fusion (IVIF) aims to integrate thermal information and detailed spatial structures into a single fused image to enhance perception.<n>Recent IVIF reward-modelling efforts learn from human ratings but use scalar regression on aggregated scores.<n>We introduce FuScore, which utilizes an MLLM to mimic human visual perception by producing continuous quality score.
  arXiv  Detail & Related papers  (2026-05-07T21:38:09Z)
• Prototype Fusion: A Training-Free Multi-Layer Approach to OOD Detection [0.8394724589095839]
  We show that intermediate layers encode equally rich and discriminative information for OOD detection.<n>We propose a model-agnostic approach that leverages internal representations across multiple layers.<n>Our approach delivers robust, architecture-agnostic performance and strong generalization for image classification.
  arXiv  Detail & Related papers  (2026-03-24T19:32:13Z)
• Predictive Sample Assignment for Semantically Coherent Out-of-Distribution Detection [62.1052001316508]
  Semantically coherent out-of-distribution detection (SCOOD) is a recently proposed realistic OOD detection setting.<n>We propose a concise SCOOD framework based on predictive sample assignment (PSA)<n>Our approach outperforms the state-of-the-art methods by a significant margin.
  arXiv  Detail & Related papers  (2025-12-15T01:18:38Z)
• GOOD: Training-Free Guided Diffusion Sampling for Out-of-Distribution Detection [61.96025941146103]
  GOOD is a novel framework that guides sampling trajectories towards OOD regions using off-the-shelf in-distribution (ID) classifiers.<n> GOOD incorporates dual-level guidance: Image-level guidance based on the gradient of log partition to reduce input likelihood, drives samples toward low-density regions in pixel space.<n>We introduce a unified OOD score that adaptively combines image and feature discrepancies, enhancing detection robustness.
  arXiv  Detail & Related papers  (2025-10-20T03:58:46Z)
• Multi-Method Ensemble for Out-of-Distribution Detection [0.0]
  We show that state-of-the-art feature truncation and scoring functions can be effectively combined.<n>We propose the Multi-Method Ensemble (MME) score, which unifies state-of-the-art OOD detectors into a single, more effective scoring function.
  arXiv  Detail & Related papers  (2025-08-29T09:47:20Z)
• Rethinking the Evaluation of Out-of-Distribution Detection: A Sorites Paradox [70.57120710151105]
  Most existing out-of-distribution (OOD) detection benchmarks classify samples with novel labels as the OOD data. Some marginal OOD samples actually have close semantic contents to the in-distribution (ID) sample, which makes determining the OOD sample a Sorites Paradox. We construct a benchmark named Incremental Shift OOD (IS-OOD) to address the issue.
  arXiv  Detail & Related papers  (2024-06-14T09:27:56Z)
• Log-Likelihood Score Level Fusion for Improved Cross-Sensor Smartphone Periocular Recognition [52.15994166413364]
  We employ fusion of several comparators to improve periocular performance when images from different smartphones are compared. We use a probabilistic fusion framework based on linear logistic regression, in which fused scores tend to be log-likelihood ratios. Our framework also provides an elegant and simple solution to handle signals from different devices, since same-sensor and cross-sensor score distributions are aligned and mapped to a common probabilistic domain.
  arXiv  Detail & Related papers  (2023-11-02T13:43:44Z)
• From Global to Local: Multi-scale Out-of-distribution Detection [129.37607313927458]
  Out-of-distribution (OOD) detection aims to detect "unknown" data whose labels have not been seen during the in-distribution (ID) training process. Recent progress in representation learning gives rise to distance-based OOD detection. We propose Multi-scale OOD DEtection (MODE), a first framework leveraging both global visual information and local region details.
  arXiv  Detail & Related papers  (2023-08-20T11:56:25Z)
• General-Purpose Multi-Modal OOD Detection Framework [5.287829685181842]
  Out-of-distribution (OOD) detection identifies test samples that differ from the training data, which is critical to ensuring the safety and reliability of machine learning (ML) systems. We propose a general-purpose weakly-supervised OOD detection framework, called WOOD, that combines a binary classifier and a contrastive learning component. We evaluate the proposed WOOD model on multiple real-world datasets, and the experimental results demonstrate that the WOOD model outperforms the state-of-the-art methods for multi-modal OOD detection.
  arXiv  Detail & Related papers  (2023-07-24T18:50:49Z)
• Improving Out-of-Distribution Detection with Disentangled Foreground and Background Features [23.266183020469065]
  We propose a novel framework that disentangles foreground and background features from ID training samples via a dense prediction approach. It is a generic framework that allows for a seamless combination with various existing OOD detection methods.
  arXiv  Detail & Related papers  (2023-03-15T16:12:14Z)
• A Multi-Scale Framework for Out-of-Distribution Detection in Dermoscopic Images [10.20384144853726]
  We propose a multi-scale detection framework to detect out-of-distribution skin disease image data. Our framework extracts features from different layers of the neural network. Experiments show that the proposed framework achieves superior performance when compared with other state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-01-18T13:49:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.