Enhancing Outlier Knowledge for Few-Shot Out-of-Distribution Detection with Extensible Local Prompts

• URL: http://arxiv.org/abs/2409.04796v1
• Date: Sat, 7 Sep 2024 11:24:52 GMT
• Title: Enhancing Outlier Knowledge for Few-Shot Out-of-Distribution Detection with Extensible Local Prompts
• Abstract summary: Out-of-Distribution (OOD) detection aims to distinguish outliers from known categories. We propose a novel tuning paradigm to emphasize regional enhancement with local prompts. Our method reduces average FPR95 by 5.17% against state-of-the-art method in 4-shot tuning on challenging ImageNet-1k dataset.
• Score: 25.51735861729728
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Out-of-Distribution (OOD) detection, aiming to distinguish outliers from known categories, has gained prominence in practical scenarios. Recently, the advent of vision-language models (VLM) has heightened interest in enhancing OOD detection for VLM through few-shot tuning. However, existing methods mainly focus on optimizing global prompts, ignoring refined utilization of local information with regard to outliers. Motivated by this, we freeze global prompts and introduce a novel coarse-to-fine tuning paradigm to emphasize regional enhancement with local prompts. Our method comprises two integral components: global prompt guided negative augmentation and local prompt enhanced regional regularization. The former utilizes frozen, coarse global prompts as guiding cues to incorporate negative augmentation, thereby leveraging local outlier knowledge. The latter employs trainable local prompts and a regional regularization to capture local information effectively, aiding in outlier identification. We also propose regional-related metric to empower the enrichment of OOD detection. Moreover, since our approach explores enhancing local prompts only, it can be seamlessly integrated with trained global prompts during inference to boost the performance. Comprehensive experiments demonstrate the effectiveness and potential of our method. Notably, our method reduces average FPR95 by 5.17% against state-of-the-art method in 4-shot tuning on challenging ImageNet-1k dataset, even outperforming 16-shot results of previous methods.

Related papers

• Morphology-optimized Multi-Scale Fusion: Combining Local Artifacts and Mesoscopic Semantics for Deepfake Detection and Localization [30.871239863769404]
  A common strategy is to incorporate forged region annotations during model training alongside manipulated images.<n>We propose a novel approach that independently predicts manipulated regions using both local and global perspectives.
  arXiv  Detail & Related papers  (2025-09-17T07:46:07Z)
• Focus on Local: Finding Reliable Discriminative Regions for Visual Place Recognition [26.27331683733832]
  Visual Place Recognition (VPR) is aimed at predicting the location of a query image by referencing a database of geotagged images. Existing methods lack precisely modeling and full exploitation of discriminative regions. We propose the Focus on Local (FoL) approach to stimulate the performance of image retrieval and re-ranking in VPR simultaneously.
  arXiv  Detail & Related papers  (2025-04-14T05:04:51Z)
• GalLoP: Learning Global and Local Prompts for Vision-Language Models [18.61218795834041]
  Global-Local Prompts (GalLoP) is a new prompt learning method that learns multiple diverse prompts leveraging both global and local visual features. GalLoP shows strong robustness performances in both domain generalization and OOD detection, even outperforming dedicated OOD detection methods.
  arXiv  Detail & Related papers  (2024-07-01T15:50:15Z)
• LoDisc: Learning Global-Local Discriminative Features for Self-Supervised Fine-Grained Visual Recognition [18.442966979622717]
  We present to incorporate the subtle local fine-grained feature learning into global self-supervised contrastive learning. A novel pretext task called Local Discrimination (LoDisc) is proposed to explicitly supervise self-supervised model's focus towards local pivotal regions. We show that Local Discrimination pretext task can effectively enhance fine-grained clues in important local regions, and the global-local framework further refines the fine-grained feature representations of images.
  arXiv  Detail & Related papers  (2024-03-06T21:36:38Z)
• From Global to Local: Multi-scale Out-of-distribution Detection [129.37607313927458]
  Out-of-distribution (OOD) detection aims to detect "unknown" data whose labels have not been seen during the in-distribution (ID) training process. Recent progress in representation learning gives rise to distance-based OOD detection. We propose Multi-scale OOD DEtection (MODE), a first framework leveraging both global visual information and local region details.
  arXiv  Detail & Related papers  (2023-08-20T11:56:25Z)
• Global Meets Local: Effective Multi-Label Image Classification via Category-Aware Weak Supervision [37.761378069277676]
  This paper builds a unified framework to perform effective noisy-proposal suppression. We develop a cross-granularity attention module to explore the complementary information between global and local features. Our framework achieves superior performance over state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-11-23T05:39:17Z)
• Adaptive Local-Component-aware Graph Convolutional Network for One-shot Skeleton-based Action Recognition [54.23513799338309]
  We present an Adaptive Local-Component-aware Graph Convolutional Network for skeleton-based action recognition. Our method provides a stronger representation than the global embedding and helps our model reach state-of-the-art.
  arXiv  Detail & Related papers  (2022-09-21T02:33:07Z)
• Delving into Sequential Patches for Deepfake Detection [64.19468088546743]
  Recent advances in face forgery techniques produce nearly untraceable deepfake videos, which could be leveraged with malicious intentions. Previous studies has identified the importance of local low-level cues and temporal information in pursuit to generalize well across deepfake methods. We propose the Local- & Temporal-aware Transformer-based Deepfake Detection framework, which adopts a local-to-global learning protocol.
  arXiv  Detail & Related papers  (2022-07-06T16:46:30Z)
• Localization Distillation for Object Detection [134.12664548771534]
  Previous knowledge distillation (KD) methods for object detection mostly focus on feature imitation instead of mimicking the classification logits. We present a novel localization distillation (LD) method which can efficiently transfer the localization knowledge from the teacher to the student. We show that logit mimicking can outperform feature imitation and the absence of localization distillation is a critical reason for why logit mimicking underperforms for years.
  arXiv  Detail & Related papers  (2022-04-12T17:14:34Z)
• Region-Based Semantic Factorization in GANs [67.90498535507106]
  We present a highly efficient algorithm to factorize the latent semantics learned by Generative Adversarial Networks (GANs) concerning an arbitrary image region. Through an appropriately defined generalized Rayleigh quotient, we solve such a problem without any annotations or training. Experimental results on various state-of-the-art GAN models demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2022-02-19T17:46:02Z)
• An Entropy-guided Reinforced Partial Convolutional Network for Zero-Shot Learning [77.72330187258498]
  We propose a novel Entropy-guided Reinforced Partial Convolutional Network (ERPCNet) ERPCNet extracts and aggregates localities based on semantic relevance and visual correlations without human-annotated regions. It not only discovers global-cooperative localities dynamically but also converges faster for policy gradient optimization.
  arXiv  Detail & Related papers  (2021-11-03T11:13:13Z)
• LoGG3D-Net: Locally Guided Global Descriptor Learning for 3D Place Recognition [31.105598103211825]
  We show that an additional training signal (local consistency loss) can guide the network to learning local features which are consistent across revisits. We formulate our approach in an end-to-end trainable architecture called LoGG3D-Net.
  arXiv  Detail & Related papers  (2021-09-17T03:32:43Z)
• Triggering Failures: Out-Of-Distribution detection by learning from local adversarial attacks in Semantic Segmentation [76.2621758731288]
  We tackle the detection of out-of-distribution (OOD) objects in semantic segmentation. Our main contribution is a new OOD detection architecture called ObsNet associated with a dedicated training scheme based on Local Adversarial Attacks (LAA) We show it obtains top performances both in speed and accuracy when compared to ten recent methods of the literature on three different datasets.
  arXiv  Detail & Related papers  (2021-08-03T17:09:56Z)
• Gait Recognition via Effective Global-Local Feature Representation and Local Temporal Aggregation [28.721376937882958]
  Gait recognition is one of the most important biometric technologies and has been applied in many fields. Recent gait recognition frameworks represent each gait frame by descriptors extracted from either global appearances or local regions of humans. We propose a novel feature extraction and fusion framework to achieve discriminative feature representations for gait recognition.
  arXiv  Detail & Related papers  (2020-11-03T04:07:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.