Bridge Feature Matching and Cross-Modal Alignment with Mutual-filtering for Zero-shot Anomaly Detection

• URL: http://arxiv.org/abs/2507.11003v1
• Date: Tue, 15 Jul 2025 05:42:17 GMT
• Title: Bridge Feature Matching and Cross-Modal Alignment with Mutual-filtering for Zero-shot Anomaly Detection
• Authors: Yuhu Bai, Jiangning Zhang, Yunkang Cao, Guangyuan Lu, Qingdong He, Xiangtai Li, Guanzhong Tian,
• Abstract summary: This study introduces textbfFiSeCLIP for ZSAD with training-free textbfCLIP, combining the feature matching with the cross-modal alignment.<n>Our approach exhibits superior performance for both anomaly classification and segmentation on anomaly detection benchmarks.
• Score: 25.349261412750586
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the advent of vision-language models (e.g., CLIP) in zero- and few-shot settings, CLIP has been widely applied to zero-shot anomaly detection (ZSAD) in recent research, where the rare classes are essential and expected in many applications. This study introduces \textbf{FiSeCLIP} for ZSAD with training-free \textbf{CLIP}, combining the feature matching with the cross-modal alignment. Testing with the entire dataset is impractical, while batch-based testing better aligns with real industrial needs, and images within a batch can serve as mutual reference points. Accordingly, FiSeCLIP utilizes other images in the same batch as reference information for the current image. However, the lack of labels for these references can introduce ambiguity, we apply text information to \textbf{fi}lter out noisy features. In addition, we further explore CLIP's inherent potential to restore its local \textbf{se}mantic correlation, adapting it for fine-grained anomaly detection tasks to enable a more accurate filtering process. Our approach exhibits superior performance for both anomaly classification and segmentation on anomaly detection benchmarks, building a stronger baseline for the direction, e.g., on MVTec-AD, FiSeCLIP outperforms the SOTA AdaCLIP by +4.6\%$\uparrow$/+5.7\%$\uparrow$ in segmentation metrics AU-ROC/$F_1$-max.

Related papers

• MadCLIP: Few-shot Medical Anomaly Detection with CLIP [14.023527193608142]
  An innovative few-shot anomaly detection approach is presented, leveraging the pre-trained CLIP model for medical data.<n>A dual-branch design is proposed to separately capture normal and abnormal features through learnable adapters.<n>To improve semantic alignment, learnable text prompts are employed to link visual features.
  arXiv  Detail & Related papers  (2025-06-30T12:56:17Z)
• Crane: Context-Guided Prompt Learning and Attention Refinement for Zero-Shot Anomaly Detections [50.343419243749054]
  Anomaly Detection (AD) involves identifying deviations from normal data distributions.<n>We propose a novel approach that conditions the prompts of the text encoder based on image context extracted from the vision encoder.<n>Our method achieves state-of-the-art performance, improving performance by 2% to 29% across different metrics on 14 datasets.
  arXiv  Detail & Related papers  (2025-04-15T10:42:25Z)
• CLIP-FSAC++: Few-Shot Anomaly Classification with Anomaly Descriptor Based on CLIP [22.850815902535988]
  We propose an effective few-shot anomaly classification framework with one-stage training, dubbed CLIP-FSAC++.<n>In anomaly descriptor, image-to-text cross-attention module is used to obtain image-specific text embeddings.<n> Comprehensive experiment results are provided for evaluating our method in few-normal shot anomaly classification on VisA and MVTEC-AD for 1, 2, 4 and 8-shot settings.
  arXiv  Detail & Related papers  (2024-12-05T02:44:45Z)
• Bridge the Points: Graph-based Few-shot Segment Anything Semantically [79.1519244940518]
  Recent advancements in pre-training techniques have enhanced the capabilities of vision foundation models. Recent studies extend the SAM to Few-shot Semantic segmentation (FSS) We propose a simple yet effective approach based on graph analysis.
  arXiv  Detail & Related papers  (2024-10-09T15:02:28Z)
• AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detection [14.916862007773341]
  This study introduces AdaCLIP for the ZSAD task, leveraging a pre-trained vision-language model (VLM), CLIP. AdaCLIP incorporates learnable prompts into CLIP and optimize them through training on auxiliary annotated anomaly detection data. Experiments conducted across 14 real-world anomaly detection datasets from industrial and medical domains indicate that AdaCLIP outperforms other ZSAD methods.
  arXiv  Detail & Related papers  (2024-07-22T16:52:37Z)
• African or European Swallow? Benchmarking Large Vision-Language Models for Fine-Grained Object Classification [53.89380284760555]
  textttFOCI (textbfFine-grained textbfObject textbfClasstextbfIfication) is a difficult multiple-choice benchmark for fine-grained object classification. textttFOCIxspace complements five popular classification datasets with four domain-specific subsets from ImageNet-21k.
  arXiv  Detail & Related papers  (2024-06-20T16:59:39Z)
• FiLo: Zero-Shot Anomaly Detection by Fine-Grained Description and High-Quality Localization [31.854923603517264]
  We propose a novel zero-shot anomaly detection (ZSAD) method called FiLo. FiLo comprises two components: adaptively learned Fine-Grained Description (FG-Des) and position-enhanced High- quality localization (HQ-Loc) Experimental results on datasets like MVTec and VisA demonstrate that FiLo significantly improves the performance of ZSAD in both detection and localization.
  arXiv  Detail & Related papers  (2024-04-21T14:22:04Z)
• Weak Supervision with Arbitrary Single Frame for Micro- and Macro-expression Spotting [22.04975008531069]
  We propose a point-level weakly-supervised expression spotting framework, where each expression requires to be annotated with only one random frame (i.e., a point) We show MPLG generates more reliable pseudo labels by merging class-specific probabilities, attention scores, fused features, and point-level labels. Experiments on the CAS(ME)2, CAS(ME)3, and SAMM-LV datasets demonstrate PWES achieves promising performance comparable to that of recent fully-supervised methods.
  arXiv  Detail & Related papers  (2024-03-21T09:01:21Z)
• Bootstrap Fine-Grained Vision-Language Alignment for Unified Zero-Shot Anomaly Localization [63.61093388441298]
  Contrastive Language-Image Pre-training models have shown promising performance on zero-shot visual recognition tasks. In this work, we propose AnoCLIP for zero-shot anomaly localization.
  arXiv  Detail & Related papers  (2023-08-30T10:35:36Z)
• Rapid Adaptation in Online Continual Learning: Are We Evaluating It Right? [135.71855998537347]
  We revisit the common practice of evaluating adaptation of Online Continual Learning (OCL) algorithms through the metric of online accuracy. We show that this metric is unreliable, as even vacuous blind classifiers can achieve unrealistically high online accuracy. Existing OCL algorithms can also achieve high online accuracy, but perform poorly in retaining useful information.
  arXiv  Detail & Related papers  (2023-05-16T08:29:33Z)
• CAFS: Class Adaptive Framework for Semi-Supervised Semantic Segmentation [5.484296906525601]
  Semi-supervised semantic segmentation learns a model for classifying pixels into specific classes using a few labeled samples and numerous unlabeled images. We propose a class-adaptive semisupervision framework for semi-supervised semantic segmentation (CAFS) CAFS constructs a validation set on a labeled dataset, to leverage the calibration performance for each class.
  arXiv  Detail & Related papers  (2023-03-21T05:56:53Z)
• Hierarchical Semi-Supervised Contrastive Learning for Contamination-Resistant Anomaly Detection [81.07346419422605]
  Anomaly detection aims at identifying deviant samples from the normal data distribution. Contrastive learning has provided a successful way to sample representation that enables effective discrimination on anomalies. We propose a novel hierarchical semi-supervised contrastive learning framework, for contamination-resistant anomaly detection.
  arXiv  Detail & Related papers  (2022-07-24T18:49:26Z)
• The CLEAR Benchmark: Continual LEArning on Real-World Imagery [77.98377088698984]
  Continual learning (CL) is widely regarded as crucial challenge for lifelong AI. We introduce CLEAR, the first continual image classification benchmark dataset with a natural temporal evolution of visual concepts. We find that a simple unsupervised pre-training step can already boost state-of-the-art CL algorithms.
  arXiv  Detail & Related papers  (2022-01-17T09:09:09Z)
• Dense Label Encoding for Boundary Discontinuity Free Rotation Detection [69.75559390700887]
  This paper explores a relatively less-studied methodology based on classification. We propose new techniques to push its frontier in two aspects. Experiments and visual analysis on large-scale public datasets for aerial images show the effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-11-19T05:42:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.