Representation Norm Amplification for Out-of-Distribution Detection in Long-Tail Learning

• URL: http://arxiv.org/abs/2408.10676v1
• Date: Tue, 20 Aug 2024 09:27:07 GMT
• Title: Representation Norm Amplification for Out-of-Distribution Detection in Long-Tail Learning
• Authors: Dong Geun Shin, Hye Won Chung,
• Abstract summary: We introduce our method, called textitRepresentation Norm Amplification (RNA), which solves the problem of detecting out-of-distribution samples. Experiments show that RNA achieves superior performance in both OOD detection and classification compared to the state-of-the-art methods.
• Score: 10.696635172502141
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Detecting out-of-distribution (OOD) samples is a critical task for reliable machine learning. However, it becomes particularly challenging when the models are trained on long-tailed datasets, as the models often struggle to distinguish tail-class in-distribution samples from OOD samples. We examine the main challenges in this problem by identifying the trade-offs between OOD detection and in-distribution (ID) classification, faced by existing methods. We then introduce our method, called \textit{Representation Norm Amplification} (RNA), which solves this challenge by decoupling the two problems. The main idea is to use the norm of the representation as a new dimension for OOD detection, and to develop a training method that generates a noticeable discrepancy in the representation norm between ID and OOD data, while not perturbing the feature learning for ID classification. Our experiments show that RNA achieves superior performance in both OOD detection and classification compared to the state-of-the-art methods, by 1.70\% and 9.46\% in FPR95 and 2.43\% and 6.87\% in classification accuracy on CIFAR10-LT and ImageNet-LT, respectively. The code for this work is available at https://github.com/dgshin21/RNA.

Related papers

• Going Beyond Conventional OOD Detection [0.0]
  Out-of-distribution (OOD) detection is critical to ensure the safe deployment of deep learning models in critical applications. We present a unified Approach to Spurimatious, fine-grained, and Conventional OOD Detection (ASCOOD) Our approach effectively mitigates the impact of spurious correlations and encourages capturing fine-grained attributes.
  arXiv  Detail & Related papers  (2024-11-16T13:04:52Z)
• Out-of-Distribution Detection in Long-Tailed Recognition with Calibrated Outlier Class Learning [24.6581764192229]
  Existing out-of-distribution (OOD) methods have shown great success on balanced datasets. OOD samples are often wrongly classified into head classes and/or tail-class samples are treated as OOD samples. We introduce a novel outlier class learning (COCL) approach, in which 1) a debiased large margin learning method is introduced in the outlier class learning to distinguish OOD samples from both head and tail classes in the representation space and 2) an outlier-class-aware logit calibration method is defined to enhance the long-tailed classification confidence.
  arXiv  Detail & Related papers  (2023-12-17T11:11:02Z)
• EAT: Towards Long-Tailed Out-of-Distribution Detection [55.380390767978554]
  This paper addresses the challenging task of long-tailed OOD detection. The main difficulty lies in distinguishing OOD data from samples belonging to the tail classes. We propose two simple ideas: (1) Expanding the in-distribution class space by introducing multiple abstention classes, and (2) Augmenting the context-limited tail classes by overlaying images onto the context-rich OOD data.
  arXiv  Detail & Related papers  (2023-12-14T13:47:13Z)
• From Global to Local: Multi-scale Out-of-distribution Detection [129.37607313927458]
  Out-of-distribution (OOD) detection aims to detect "unknown" data whose labels have not been seen during the in-distribution (ID) training process. Recent progress in representation learning gives rise to distance-based OOD detection. We propose Multi-scale OOD DEtection (MODE), a first framework leveraging both global visual information and local region details.
  arXiv  Detail & Related papers  (2023-08-20T11:56:25Z)
• Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection Capability [70.72426887518517]
  Out-of-distribution (OOD) detection is an indispensable aspect of secure AI when deploying machine learning models in real-world applications. We propose a novel method, Unleashing Mask, which aims to restore the OOD discriminative capabilities of the well-trained model with ID data. Our method utilizes a mask to figure out the memorized atypical samples, and then finetune the model or prune it with the introduced mask to forget them.
  arXiv  Detail & Related papers  (2023-06-06T14:23:34Z)
• Partial and Asymmetric Contrastive Learning for Out-of-Distribution Detection in Long-Tailed Recognition [80.07843757970923]
  We show that existing OOD detection methods suffer from significant performance degradation when the training set is long-tail distributed. We propose Partial and Asymmetric Supervised Contrastive Learning (PASCL), which explicitly encourages the model to distinguish between tail-class in-distribution samples and OOD samples. Our method outperforms previous state-of-the-art method by $1.29%$, $1.45%$, $0.69%$ anomaly detection false positive rate (FPR) and $3.24%$, $4.06%$, $7.89%$ in-distribution
  arXiv  Detail & Related papers  (2022-07-04T01:53:07Z)
• Breaking Down Out-of-Distribution Detection: Many Methods Based on OOD Training Data Estimate a Combination of the Same Core Quantities [104.02531442035483]
  The goal of this paper is to recognize common objectives as well as to identify the implicit scoring functions of different OOD detection methods. We show that binary discrimination between in- and (different) out-distributions is equivalent to several distinct formulations of the OOD detection problem. We also show that the confidence loss which is used by Outlier Exposure has an implicit scoring function which differs in a non-trivial fashion from the theoretically optimal scoring function.
  arXiv  Detail & Related papers  (2022-06-20T16:32:49Z)
• Training OOD Detectors in their Natural Habitats [31.565635192716712]
  Out-of-distribution (OOD) detection is important for machine learning models deployed in the wild. Recent methods use auxiliary outlier data to regularize the model for improved OOD detection. We propose a novel framework that leverages wild mixture data -- that naturally consists of both ID and OOD samples.
  arXiv  Detail & Related papers  (2022-02-07T15:38:39Z)
• Learn what you can't learn: Regularized Ensembles for Transductive Out-of-distribution Detection [76.39067237772286]
  We show that current out-of-distribution (OOD) detection algorithms for neural networks produce unsatisfactory results in a variety of OOD detection scenarios. This paper studies how such "hard" OOD scenarios can benefit from adjusting the detection method after observing a batch of the test data. We propose a novel method that uses an artificial labeling scheme for the test data and regularization to obtain ensembles of models that produce contradictory predictions only on the OOD samples in a test batch.
  arXiv  Detail & Related papers  (2020-12-10T16:55:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.