Bayesian Prompt Flow Learning for Zero-Shot Anomaly Detection

• URL: http://arxiv.org/abs/2503.10080v2
• Date: Mon, 24 Mar 2025 00:51:39 GMT
• Title: Bayesian Prompt Flow Learning for Zero-Shot Anomaly Detection
• Authors: Zhen Qu, Xian Tao, Xinyi Gong, Shichen Qu, Qiyu Chen, Zhengtao Zhang, Xingang Wang, Guiguang Ding,
• Abstract summary: vision-language models (e.g. CLIP) have demonstrated remarkable performance in zero-shot anomaly detection (ZSAD)<n>Bayes-PFL is designed to learn both image-specific and image-agnostic distributions, which are jointly utilized to regularize the text prompt space and improve the model's generalization on unseen categories.<n>Experiments on 15 industrial and medical datasets demonstrate our method's superior performance.
• Score: 17.590853105242864
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, vision-language models (e.g. CLIP) have demonstrated remarkable performance in zero-shot anomaly detection (ZSAD). By leveraging auxiliary data during training, these models can directly perform cross-category anomaly detection on target datasets, such as detecting defects on industrial product surfaces or identifying tumors in organ tissues. Existing approaches typically construct text prompts through either manual design or the optimization of learnable prompt vectors. However, these methods face several challenges: 1) handcrafted prompts require extensive expert knowledge and trial-and-error; 2) single-form learnable prompts struggle to capture complex anomaly semantics; and 3) an unconstrained prompt space limits generalization to unseen categories. To address these issues, we propose Bayesian Prompt Flow Learning (Bayes-PFL), which models the prompt space as a learnable probability distribution from a Bayesian perspective. Specifically, a prompt flow module is designed to learn both image-specific and image-agnostic distributions, which are jointly utilized to regularize the text prompt space and improve the model's generalization on unseen categories. These learned distributions are then sampled to generate diverse text prompts, effectively covering the prompt space. Additionally, a residual cross-model attention (RCA) module is introduced to better align dynamic text embeddings with fine-grained image features. Extensive experiments on 15 industrial and medical datasets demonstrate our method's superior performance. The code is available at https://github.com/xiaozhen228/Bayes-PFL.

Related papers

• FiLo++: Zero-/Few-Shot Anomaly Detection by Fused Fine-Grained Descriptions and Deformable Localization [28.994585945398754]
  Anomaly detection methods typically require extensive normal samples from the target class for training. Existing zero-shot and few-shot approaches often leverage powerful multimodal models to detect and localize anomalies. This paper proposes the FiLo++ method, which consists of two key components.
  arXiv  Detail & Related papers  (2025-01-17T09:38:43Z)
• Prompt Diffusion Robustifies Any-Modality Prompt Learning [43.35989471544046]
  This paper introduces prompt diffusion, which uses a diffusion model to gradually refine the prompts to obtain a customized prompt for each sample. For all prompt learning methods tested, adding prompt diffusion yields more robust results for base-to-new generalization, cross-dataset generalization, and domain generalization in classification tasks tested over 15 diverse datasets.
  arXiv  Detail & Related papers  (2024-10-26T12:36:25Z)
• Dude: Dual Distribution-Aware Context Prompt Learning For Large Vision-Language Model [27.56988000960972]
  We introduce a new framework based on a dual context of both domain-shared and class-specific contexts. Such dual prompt methods enhance the model's feature representation by joining implicit and explicit factors encoded in Large Language Models. We also formulate the Unbalanced Optimal Transport (UOT) theory to quantify the relationships between constructed prompts and visual tokens.
  arXiv  Detail & Related papers  (2024-07-05T13:15:29Z)
• PromptAD: Learning Prompts with only Normal Samples for Few-Shot Anomaly Detection [59.34973469354926]
  This paper proposes a one-class prompt learning method for few-shot anomaly detection, termed PromptAD. For image-level/pixel-level anomaly detection, PromptAD achieves first place in 11/12 few-shot settings on MVTec and VisA.
  arXiv  Detail & Related papers  (2024-04-08T06:53:30Z)
• Random Word Data Augmentation with CLIP for Zero-Shot Anomaly Detection [3.75292409381511]
  This paper presents a novel method that leverages a visual-language model, CLIP, as a data source for zero-shot anomaly detection. Using the generated embeddings as training data, a feed-forward neural network learns to extract features of normal and anomaly from CLIP's embeddings. Experimental results demonstrate that our method achieves state-of-the-art performance without laborious prompt ensembling in zero-shot setups.
  arXiv  Detail & Related papers  (2023-08-22T01:55:03Z)
• Self-regulating Prompts: Foundational Model Adaptation without Forgetting [112.66832145320434]
  We introduce a self-regularization framework for prompting called PromptSRC. PromptSRC guides the prompts to optimize for both task-specific and task-agnostic general representations.
  arXiv  Detail & Related papers  (2023-07-13T17:59:35Z)
• Discovering Failure Modes of Text-guided Diffusion Models via Adversarial Search [52.519433040005126]
  Text-guided diffusion models (TDMs) are widely applied but can fail unexpectedly. In this work, we aim to study and understand the failure modes of TDMs in more detail. We propose SAGE, the first adversarial search method on TDMs.
  arXiv  Detail & Related papers  (2023-06-01T17:59:00Z)
• Hard-normal Example-aware Template Mutual Matching for Industrial Anomaly Detection [78.734927709231]
  Anomaly detectors are widely used in industrial manufacturing to detect and localize unknown defects in query images.<n>These detectors are trained on anomaly-free samples and have successfully distinguished anomalies from most normal samples.<n>However, hard-normal examples are scattered and far apart from most normal samples, and thus they are often mistaken for anomalies by existing methods.
  arXiv  Detail & Related papers  (2023-03-28T17:54:56Z)
• Patch-Prompt Aligned Bayesian Prompt Tuning for Vision-Language Models [48.77653835765705]
  We introduce a probabilistic resolution to prompt tuning, where the label-specific prompts are generated hierarchically by first sampling a latent vector from an underlying distribution and then employing a lightweight generative model. We evaluate the effectiveness of our approach on four tasks: few-shot image recognition, base-to-new generalization, dataset transfer learning, and domain shifts.
  arXiv  Detail & Related papers  (2023-03-16T06:09:15Z)
• Bayesian Prompt Learning for Image-Language Model Generalization [64.50204877434878]
  We use the regularization ability of Bayesian methods to frame prompt learning as a variational inference problem. Our approach regularizes the prompt space, reduces overfitting to the seen prompts and improves the prompt generalization on unseen prompts. We demonstrate empirically on 15 benchmarks that Bayesian prompt learning provides an appropriate coverage of the prompt space.
  arXiv  Detail & Related papers  (2022-10-05T17:05:56Z)
• LASP: Text-to-Text Optimization for Language-Aware Soft Prompting of Vision & Language Models [67.19124099815645]
  We propose a novel Language-Aware Soft Prompting (LASP) learning method to alleviate base class overfitting. LASP is inherently amenable to including, during training, virtual classes, i.e. class names for which no visual samples are available. LASP matches and surpasses, for the first time, the accuracy on novel classes obtained by hand-crafted prompts and CLIP for 8 out of 11 test datasets.
  arXiv  Detail & Related papers  (2022-10-03T17:56:35Z)
• PromptDet: Expand Your Detector Vocabulary with Uncurated Images [47.600059694034]
  The goal of this work is to establish a scalable pipeline for expanding an object detector towards novel/unseen categories, using zero manual annotations. We propose a two-stage open-vocabulary object detector that categorises each box proposal by a classifier generated from the text encoder of a pre-trained visual-language model. To scale up the learning procedure towards detecting a wider spectrum of objects, we exploit the available online resource, iteratively updating the prompts, and later self-training the proposed detector with pseudo labels generated on a large corpus of noisy, uncurated web images.
  arXiv  Detail & Related papers  (2022-03-30T17:50:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.