Segment Any Anomaly without Training via Hybrid Prompt Regularization

• URL: http://arxiv.org/abs/2305.10724v1
• Date: Thu, 18 May 2023 05:52:06 GMT
• Title: Segment Any Anomaly without Training via Hybrid Prompt Regularization
• Authors: Yunkang Cao, Xiaohao Xu, Chen Sun, Yuqi Cheng, Zongwei Du, Liang Gao, Weiming Shen
• Abstract summary: We present a novel framework, i.e., Segment Any Anomaly + (SAA+), for zero-shot anomaly segmentation with hybrid prompt regularization. Our proposed SAA+ model achieves state-of-the-art performance on several anomaly segmentation benchmarks, including VisA, MVTec-AD, MTD, and KSDD2.
• Score: 15.38935129648466
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: We present a novel framework, i.e., Segment Any Anomaly + (SAA+), for zero-shot anomaly segmentation with hybrid prompt regularization to improve the adaptability of modern foundation models. Existing anomaly segmentation models typically rely on domain-specific fine-tuning, limiting their generalization across countless anomaly patterns. In this work, inspired by the great zero-shot generalization ability of foundation models like Segment Anything, we first explore their assembly to leverage diverse multi-modal prior knowledge for anomaly localization. For non-parameter foundation model adaptation to anomaly segmentation, we further introduce hybrid prompts derived from domain expert knowledge and target image context as regularization. Our proposed SAA+ model achieves state-of-the-art performance on several anomaly segmentation benchmarks, including VisA, MVTec-AD, MTD, and KSDD2, in the zero-shot setting. We will release the code at \href{https://github.com/caoyunkang/Segment-Any-Anomaly}{https://github.com/caoyunkang/Segment-Any-Anomaly}.

Related papers

• Generalize or Detect? Towards Robust Semantic Segmentation Under Multiple Distribution Shifts [56.57141696245328]
  In open-world scenarios, where both novel classes and domains may exist, an ideal segmentation model should detect anomaly classes for safety. Existing methods often struggle to distinguish between domain-level and semantic-level distribution shifts.
  arXiv  Detail & Related papers  (2024-11-06T11:03:02Z)
• The Non-Local Model Merging Problem: Permutation Symmetries and Variance Collapse [25.002218722102505]
  Model merging aims to efficiently combine the weights of multiple expert models, each trained on a specific task, into a single multi-task model. This work explores the more challenging scenario of "non-local" merging. Standard merging techniques often fail to generalize effectively in this non-local setting. We propose a multi-task technique to re-scale and shift the output activations of the merged model for each task, aligning its output statistics with those of the corresponding task-specific expert models.
  arXiv  Detail & Related papers  (2024-10-16T17:41:59Z)
• Prompting Segment Anything Model with Domain-Adaptive Prototype for Generalizable Medical Image Segmentation [49.5901368256326]
  We propose a novel Domain-Adaptive Prompt framework for fine-tuning the Segment Anything Model (termed as DAPSAM) in segmenting medical images. Our DAPSAM achieves state-of-the-art performance on two medical image segmentation tasks with different modalities.
  arXiv  Detail & Related papers  (2024-09-19T07:28:33Z)
• Collaborating Foundation Models for Domain Generalized Semantic Segmentation [23.359941294938142]
  Domain Generalized Semantic (DGSS) deals with training a model on a labeled source domain. We take an approach to DGSS and propose to use an assembly of CoLlaborative FOUndation models for Domain Generalized Semantic (CLOUDS)
  arXiv  Detail & Related papers  (2023-12-15T13:43:24Z)
• UGainS: Uncertainty Guided Anomaly Instance Segmentation [80.12253291709673]
  A single unexpected object on the road can cause an accident or lead to injuries. Current approaches tackle anomaly segmentation by assigning an anomaly score to each pixel. We propose an approach that produces accurate anomaly instance masks.
  arXiv  Detail & Related papers  (2023-08-03T20:55:37Z)
• 2nd Place Winning Solution for the CVPR2023 Visual Anomaly and Novelty Detection Challenge: Multimodal Prompting for Data-centric Anomaly Detection [10.682758791557436]
  This report introduces the winning solution of the team Segment Any Anomaly for the CVPR2023 Visual Anomaly and Novelty Detection (VAND) challenge. We present a novel framework, i.e., Segment Any Anomaly + (SAA$+$), for zero-shot anomaly segmentation with multi-modal prompts. We will release the code of our winning solution for the CVPR2023 VAN.
  arXiv  Detail & Related papers  (2023-06-15T11:49:44Z)
• Decoupled Multi-task Learning with Cyclical Self-Regulation for Face Parsing [71.19528222206088]
  We propose a novel Decoupled Multi-task Learning with Cyclical Self-Regulation for face parsing. Specifically, DML-CSR designs a multi-task model which comprises face parsing, binary edge, and category edge detection. Our method achieves the new state-of-the-art performance on the Helen, CelebA-HQ, and LapaMask datasets.
  arXiv  Detail & Related papers  (2022-03-28T02:12:30Z)
• Correlation Clustering Reconstruction in Semi-Adversarial Models [70.11015369368272]
  Correlation Clustering is an important clustering problem with many applications. We study the reconstruction version of this problem in which one is seeking to reconstruct a latent clustering corrupted by random noise and adversarial modifications.
  arXiv  Detail & Related papers  (2021-08-10T14:46:17Z)
• Posterior Differential Regularization with f-divergence for Improving Model Robustness [95.05725916287376]
  We focus on methods that regularize the model posterior difference between clean and noisy inputs. We generalize the posterior differential regularization to the family of $f$-divergences. Our experiments show that regularizing the posterior differential with $f$-divergence can result in well-improved model robustness.
  arXiv  Detail & Related papers  (2020-10-23T19:58:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.