Activation Subspaces for Out-of-Distribution Detection

• URL: http://arxiv.org/abs/2508.21695v1
• Date: Fri, 29 Aug 2025 15:03:36 GMT
• Title: Activation Subspaces for Out-of-Distribution Detection
• Abstract summary: Out-of-distribution (OOD) detection methods aim to distinguish samples close to the training distribution from those farther away.<n>We propose a novel OOD detection method that utilizes singular value decomposition of the weight matrix of the classification head.<n>We find that the subspace of insignificant components more effectively distinguishes ID from OOD data than raw activations in regimes of large distribution shifts.
• Score: 11.782674956998939
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To ensure the reliability of deep models in real-world applications, out-of-distribution (OOD) detection methods aim to distinguish samples close to the training distribution (in-distribution, ID) from those farther away (OOD). In this work, we propose a novel OOD detection method that utilizes singular value decomposition of the weight matrix of the classification head to decompose the model's activations into decisive and insignificant components, which contribute maximally, respectively minimally, to the final classifier output. We find that the subspace of insignificant components more effectively distinguishes ID from OOD data than raw activations in regimes of large distribution shifts (Far-OOD). This occurs because the classification objective leaves the insignificant subspace largely unaffected, yielding features that are ''untainted'' by the target classification task. Conversely, in regimes of smaller distribution shifts (Near-OOD), we find that activation shaping methods profit from only considering the decisive subspace, as the insignificant component can cause interference in the activation space. By combining two findings into a single approach, termed ActSub, we achieve state-of-the-art results in various standard OOD benchmarks.

Related papers

• Unified Multi-Layer Subspace Modeling for Cross-Domain OOD Detection [12.06155749281499]
  PRISM is a model-agnostic post-hoc OOD detection method.<n>It fuses intermediate and deep features into a single hierarchical embedding.<n>It captures both in-subspace semantic deviations and off-subspace anomalies.
  arXiv  Detail & Related papers  (2026-09-06T19:00:54Z)
• Sparse Autoencoders for Interpretable Out-of-Distribution Detection [5.034534322537894]
  This paper introduces a novel approach that leverages sparse autoencoders (SAEs) to learn interpretable features from intermediate activations.<n>We propose a new OOD score derived from the cosine similarity between the sparse feature activations of a test sample and the mean activations of ID classes.
  arXiv  Detail & Related papers  (2026-07-13T19:13:40Z)
• Perturbations in the Orthogonal Complement Subspace for Efficient Out-of-Distribution Detection [5.986846311786858]
  Out-of-distribution (OOD) detection is essential for deploying deep learning models in open-world environments.<n>We introduce P-OCS, a lightweight and theoretically grounded method that operates in the complement of the principal subspace defined by ID features.<n>We show that a one-step update is sufficient in the small-perturbation regime and provide convergence guarantees for the resulting detection score.
  arXiv  Detail & Related papers  (2025-11-02T08:21:13Z)
• Revisiting Logit Distributions for Reliable Out-of-Distribution Detection [73.9121001113687]
  Out-of-distribution (OOD) detection is critical for ensuring the reliability of deep learning models in open-world applications.<n>LogitGap is a novel post-hoc OOD detection method that exploits the relationship between the maximum logit and the remaining logits.<n>We show that LogitGap consistently achieves state-of-the-art performance across diverse OOD detection scenarios and benchmarks.
  arXiv  Detail & Related papers  (2025-10-23T02:16:45Z)
• GOOD: Training-Free Guided Diffusion Sampling for Out-of-Distribution Detection [61.96025941146103]
  GOOD is a novel framework that guides sampling trajectories towards OOD regions using off-the-shelf in-distribution (ID) classifiers.<n> GOOD incorporates dual-level guidance: Image-level guidance based on the gradient of log partition to reduce input likelihood, drives samples toward low-density regions in pixel space.<n>We introduce a unified OOD score that adaptively combines image and feature discrepancies, enhancing detection robustness.
  arXiv  Detail & Related papers  (2025-10-20T03:58:46Z)
• Pseudo-label Induced Subspace Representation Learning for Robust Out-of-Distribution Detection [6.5679810906772325]
  We propose a novel OOD detection framework based on a pseudo-label-induced subspace representation.<n>In addition, we introduce a simple yet effective learning criterion that integrates a cross-entropy-based ID classification loss with a subspace distance-based regularization loss to enhance ID-OOD separability.
  arXiv  Detail & Related papers  (2025-08-05T05:38:00Z)
• Semantic or Covariate? A Study on the Intractable Case of Out-of-Distribution Detection [70.57120710151105]
  We provide a more precise definition of the Semantic Space for the ID distribution. We also define the "Tractable OOD" setting which ensures the distinguishability of OOD and ID distributions.
  arXiv  Detail & Related papers  (2024-11-18T03:09:39Z)
• What If the Input is Expanded in OOD Detection? [77.37433624869857]
  Out-of-distribution (OOD) detection aims to identify OOD inputs from unknown classes. Various scoring functions are proposed to distinguish it from in-distribution (ID) data. We introduce a novel perspective, i.e., employing different common corruptions on the input space.
  arXiv  Detail & Related papers  (2024-10-24T06:47:28Z)
• Margin-bounded Confidence Scores for Out-of-Distribution Detection [2.373572816573706]
  We propose a novel method called Margin bounded Confidence Scores (MaCS) to address the nontrivial OOD detection problem. MaCS enlarges the disparity between ID and OOD scores, which in turn makes the decision boundary more compact. Experiments on various benchmark datasets for image classification tasks demonstrate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2024-09-22T05:40:25Z)
• GROOD: Gradient-Aware Out-of-Distribution Detection [11.862922321532817]
  Out-of-distribution (OOD) detection is crucial for ensuring the reliability of deep learning models in real-world applications.<n>We propose GRadient-aware Out-Of-Distribution detection (GROOD), a method that derives an OOD prototype from synthetic samples and computes class prototypes directly from In-distribution (ID) training data.<n>By analyzing the gradients of a nearest-class-prototype loss function concerning an artificial OOD prototype, our approach achieves a clear separation between in-distribution and OOD samples.
  arXiv  Detail & Related papers  (2023-12-22T04:28:43Z)
• LINe: Out-of-Distribution Detection by Leveraging Important Neurons [15.797257361788812]
  We introduce a new aspect for analyzing the difference in model outputs between in-distribution data and OOD data. We propose a novel method, Leveraging Important Neurons (LINe), for post-hoc Out of distribution detection.
  arXiv  Detail & Related papers  (2023-03-24T13:49:05Z)
• Breaking Down Out-of-Distribution Detection: Many Methods Based on OOD Training Data Estimate a Combination of the Same Core Quantities [104.02531442035483]
  The goal of this paper is to recognize common objectives as well as to identify the implicit scoring functions of different OOD detection methods. We show that binary discrimination between in- and (different) out-distributions is equivalent to several distinct formulations of the OOD detection problem. We also show that the confidence loss which is used by Outlier Exposure has an implicit scoring function which differs in a non-trivial fashion from the theoretically optimal scoring function.
  arXiv  Detail & Related papers  (2022-06-20T16:32:49Z)
• How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection? [22.519572587827213]
  CIDER is a representation learning framework that exploits hyperspherical embeddings for OOD detection. CIDER establishes superior performance, outperforming the latest rival by 19.36% in FPR95.
  arXiv  Detail & Related papers  (2022-03-08T23:44:01Z)
• Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic Uncertainty [58.144520501201995]
  Bi-Lipschitz regularization of neural network layers preserve relative distances between data instances in the feature spaces of each layer. With the use of an attentive set encoder, we propose to meta learn either diagonal or diagonal plus low-rank factors to efficiently construct task specific covariance matrices. We also propose an inference procedure which utilizes scaled energy to achieve a final predictive distribution.
  arXiv  Detail & Related papers  (2021-10-12T22:04:19Z)
• Triggering Failures: Out-Of-Distribution detection by learning from local adversarial attacks in Semantic Segmentation [76.2621758731288]
  We tackle the detection of out-of-distribution (OOD) objects in semantic segmentation. Our main contribution is a new OOD detection architecture called ObsNet associated with a dedicated training scheme based on Local Adversarial Attacks (LAA) We show it obtains top performances both in speed and accuracy when compared to ten recent methods of the literature on three different datasets.
  arXiv  Detail & Related papers  (2021-08-03T17:09:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.