Segment Any Anomaly without Training via Hybrid Prompt Regularization
- URL: http://arxiv.org/abs/2305.10724v1
- Date: Thu, 18 May 2023 05:52:06 GMT
- Title: Segment Any Anomaly without Training via Hybrid Prompt Regularization
- Authors: Yunkang Cao, Xiaohao Xu, Chen Sun, Yuqi Cheng, Zongwei Du, Liang Gao,
Weiming Shen
- Abstract summary: We present a novel framework, i.e., Segment Any Anomaly + (SAA+), for zero-shot anomaly segmentation with hybrid prompt regularization.
Our proposed SAA+ model achieves state-of-the-art performance on several anomaly segmentation benchmarks, including VisA, MVTec-AD, MTD, and KSDD2.
- Score: 15.38935129648466
- License: http://creativecommons.org/publicdomain/zero/1.0/
- Abstract: We present a novel framework, i.e., Segment Any Anomaly + (SAA+), for
zero-shot anomaly segmentation with hybrid prompt regularization to improve the
adaptability of modern foundation models. Existing anomaly segmentation models
typically rely on domain-specific fine-tuning, limiting their generalization
across countless anomaly patterns. In this work, inspired by the great
zero-shot generalization ability of foundation models like Segment Anything, we
first explore their assembly to leverage diverse multi-modal prior knowledge
for anomaly localization. For non-parameter foundation model adaptation to
anomaly segmentation, we further introduce hybrid prompts derived from domain
expert knowledge and target image context as regularization. Our proposed SAA+
model achieves state-of-the-art performance on several anomaly segmentation
benchmarks, including VisA, MVTec-AD, MTD, and KSDD2, in the zero-shot setting.
We will release the code at
\href{https://github.com/caoyunkang/Segment-Any-Anomaly}{https://github.com/caoyunkang/Segment-Any-Anomaly}.
Related papers
- Collaborating Foundation Models for Domain Generalized Semantic Segmentation [23.359941294938142]
Domain Generalized Semantic (DGSS) deals with training a model on a labeled source domain.
We take an approach to DGSS and propose to use an assembly of CoLlaborative FOUndation models for Domain Generalized Semantic (CLOUDS)
arXiv Detail & Related papers (2023-12-15T13:43:24Z) - UGainS: Uncertainty Guided Anomaly Instance Segmentation [80.12253291709673]
A single unexpected object on the road can cause an accident or lead to injuries.
Current approaches tackle anomaly segmentation by assigning an anomaly score to each pixel.
We propose an approach that produces accurate anomaly instance masks.
arXiv Detail & Related papers (2023-08-03T20:55:37Z) - 2nd Place Winning Solution for the CVPR2023 Visual Anomaly and Novelty
Detection Challenge: Multimodal Prompting for Data-centric Anomaly Detection [10.682758791557436]
This report introduces the winning solution of the team Segment Any Anomaly for the CVPR2023 Visual Anomaly and Novelty Detection (VAND) challenge.
We present a novel framework, i.e., Segment Any Anomaly + (SAA$+$), for zero-shot anomaly segmentation with multi-modal prompts.
We will release the code of our winning solution for the CVPR2023 VAN.
arXiv Detail & Related papers (2023-06-15T11:49:44Z) - Semantic-Aware Domain Generalized Segmentation [67.49163582961877]
Deep models trained on source domain lack generalization when evaluated on unseen target domains with different data distributions.
We propose a framework including two novel modules: Semantic-Aware Normalization (SAN) and Semantic-Aware Whitening (SAW)
Our approach shows significant improvements over existing state-of-the-art on various backbone networks.
arXiv Detail & Related papers (2022-04-02T09:09:59Z) - Decoupled Multi-task Learning with Cyclical Self-Regulation for Face
Parsing [71.19528222206088]
We propose a novel Decoupled Multi-task Learning with Cyclical Self-Regulation for face parsing.
Specifically, DML-CSR designs a multi-task model which comprises face parsing, binary edge, and category edge detection.
Our method achieves the new state-of-the-art performance on the Helen, CelebA-HQ, and LapaMask datasets.
arXiv Detail & Related papers (2022-03-28T02:12:30Z) - Correlation Clustering Reconstruction in Semi-Adversarial Models [70.11015369368272]
Correlation Clustering is an important clustering problem with many applications.
We study the reconstruction version of this problem in which one is seeking to reconstruct a latent clustering corrupted by random noise and adversarial modifications.
arXiv Detail & Related papers (2021-08-10T14:46:17Z) - Posterior Differential Regularization with f-divergence for Improving
Model Robustness [95.05725916287376]
We focus on methods that regularize the model posterior difference between clean and noisy inputs.
We generalize the posterior differential regularization to the family of $f$-divergences.
Our experiments show that regularizing the posterior differential with $f$-divergence can result in well-improved model robustness.
arXiv Detail & Related papers (2020-10-23T19:58:01Z) - Feature Transformation Ensemble Model with Batch Spectral Regularization
for Cross-Domain Few-Shot Classification [66.91839845347604]
We propose an ensemble prediction model by performing diverse feature transformations after a feature extraction network.
We use a batch spectral regularization term to suppress the singular values of the feature matrix during pre-training to improve the generalization ability of the model.
The proposed model can then be fine tuned in the target domain to address few-shot classification.
arXiv Detail & Related papers (2020-05-18T05:31:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.