Conjugated Semantic Pool Improves OOD Detection with Pre-trained Vision-Language Models

• URL: http://arxiv.org/abs/2410.08611v1
• Date: Fri, 11 Oct 2024 08:24:11 GMT
• Title: Conjugated Semantic Pool Improves OOD Detection with Pre-trained Vision-Language Models
• Authors: Mengyuan Chen, Junyu Gao, Changsheng Xu,
• Abstract summary: A straightforward pipeline for zero-shot out-of-distribution (OOD) detection involves selecting potential OOD labels from an extensive semantic pool. We theorize that enhancing performance requires expanding the semantic pool. We show that expanding OOD label candidates with the CSP satisfies the requirements and outperforms existing works by 7.89% in FPR95.
• Score: 70.82728812001807
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A straightforward pipeline for zero-shot out-of-distribution (OOD) detection involves selecting potential OOD labels from an extensive semantic pool and then leveraging a pre-trained vision-language model to perform classification on both in-distribution (ID) and OOD labels. In this paper, we theorize that enhancing performance requires expanding the semantic pool, while increasing the expected probability of selected OOD labels being activated by OOD samples, and ensuring low mutual dependence among the activations of these OOD labels. A natural expansion manner is to adopt a larger lexicon; however, the inevitable introduction of numerous synonyms and uncommon words fails to meet the above requirements, indicating that viable expansion manners move beyond merely selecting words from a lexicon. Since OOD detection aims to correctly classify input images into ID/OOD class groups, we can "make up" OOD label candidates which are not standard class names but beneficial for the process. Observing that the original semantic pool is comprised of unmodified specific class names, we correspondingly construct a conjugated semantic pool (CSP) consisting of modified superclass names, each serving as a cluster center for samples sharing similar properties across different categories. Consistent with our established theory, expanding OOD label candidates with the CSP satisfies the requirements and outperforms existing works by 7.89% in FPR95. Codes are available in https://github.com/MengyuanChen21/NeurIPS2024-CSP.

Related papers

• Diversity-grounded Channel Prototypical Learning for Out-of-Distribution Intent Detection [18.275098909064127]
  This study presents a novel fine-tuning framework for large language models (LLMs) We construct semantic prototypes for each ID class using a diversity-grounded prompt tuning approach. For a thorough assessment, we benchmark our method against the prevalent fine-tuning approaches.
  arXiv  Detail & Related papers  (2024-09-17T12:07:17Z)
• Rethinking the Evaluation of Out-of-Distribution Detection: A Sorites Paradox [70.57120710151105]
  Most existing out-of-distribution (OOD) detection benchmarks classify samples with novel labels as the OOD data. Some marginal OOD samples actually have close semantic contents to the in-distribution (ID) sample, which makes determining the OOD sample a Sorites Paradox. We construct a benchmark named Incremental Shift OOD (IS-OOD) to address the issue.
  arXiv  Detail & Related papers  (2024-06-14T09:27:56Z)
• Zero-Shot Out-of-Distribution Detection with Outlier Label Exposure [23.266183020469065]
  Outlier Label Exposure (OLE) is an approach to enhance zero-shot OOD detection using auxiliary outlier class labels. OLE substantially improves detection performance and achieves new state-of-the-art performance in large-scale OOD and hard OOD detection benchmarks.
  arXiv  Detail & Related papers  (2024-06-03T10:07:21Z)
• Envisioning Outlier Exposure by Large Language Models for Out-of-Distribution Detection [71.93411099797308]
  Out-of-distribution (OOD) samples are crucial when deploying machine learning models in open-world scenarios. We propose to tackle this constraint by leveraging the expert knowledge and reasoning capability of large language models (LLM) to potential Outlier Exposure, termed EOE. EOE can be generalized to different tasks, including far, near, and fine-language OOD detection. EOE achieves state-of-the-art performance across different OOD tasks and can be effectively scaled to the ImageNet-1K dataset.
  arXiv  Detail & Related papers  (2024-06-02T17:09:48Z)
• CLIPScope: Enhancing Zero-Shot OOD Detection with Bayesian Scoring [16.0716584170549]
  We introduce CLIPScope, a zero-shot OOD detection approach that normalizes the confidence score of a sample by class likelihoods. CLIPScope incorporates a novel strategy to mine OOD classes from a large lexical database. It selects class labels that are farthest and nearest to ID classes in terms of CLIP embedding distance to maximize coverage of OOD samples.
  arXiv  Detail & Related papers  (2024-05-23T16:03:55Z)
• Negative Label Guided OOD Detection with Pretrained Vision-Language Models [96.67087734472912]
  Out-of-distribution (OOD) detection aims at identifying samples from unknown classes. We propose a novel post hoc OOD detection method, called NegLabel, which takes a vast number of negative labels from extensive corpus databases.
  arXiv  Detail & Related papers  (2024-03-29T09:19:52Z)
• How Does Fine-Tuning Impact Out-of-Distribution Detection for Vision-Language Models? [29.75562085178755]
  We study how fine-tuning impact OOD detection for few-shot downstream tasks. Our results suggest that a proper choice of OOD scores is essential for CLIP-based fine-tuning. We also show that prompt learning demonstrates the state-of-the-art OOD detection performance over the zero-shot counterpart.
  arXiv  Detail & Related papers  (2023-06-09T17:16:50Z)
• Out-of-Distributed Semantic Pruning for Robust Semi-Supervised Learning [17.409939628100517]
  We propose a unified framework termed OOD Semantic Pruning (OSP), which aims at pruning OOD semantics out from in-distribution (ID) features. OSP surpasses the previous state-of-the-art by 13.7% on accuracy for ID classification and 5.9% on AUROC for OOD detection on TinyImageNet dataset.
  arXiv  Detail & Related papers  (2023-05-29T15:37:07Z)
• Estimating Soft Labels for Out-of-Domain Intent Detection [122.68266151023676]
  Out-of-Domain (OOD) intent detection is important for practical dialog systems. We propose an adaptive soft pseudo labeling (ASoul) method that can estimate soft labels for pseudo OOD samples.
  arXiv  Detail & Related papers  (2022-11-10T13:31:13Z)
• MASKER: Masked Keyword Regularization for Reliable Text Classification [73.90326322794803]
  We propose a fine-tuning method, coined masked keyword regularization (MASKER), that facilitates context-based prediction. MASKER regularizes the model to reconstruct the keywords from the rest of the words and make low-confidence predictions without enough context. We demonstrate that MASKER improves OOD detection and cross-domain generalization without degrading classification accuracy.
  arXiv  Detail & Related papers  (2020-12-17T04:54:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.