SAM-LAD: Segment Anything Model Meets Zero-Shot Logic Anomaly Detection

• URL: http://arxiv.org/abs/2406.00625v3
• Date: Sat, 14 Sep 2024 09:33:18 GMT
• Title: SAM-LAD: Segment Anything Model Meets Zero-Shot Logic Anomaly Detection
• Authors: Yun Peng, Xiao Lin, Nachuan Ma, Jiayuan Du, Chuangwei Liu, Chengju Liu, Qijun Chen,
• Abstract summary: Visual anomaly detection is vital in real-world applications, such as industrial defect detection and medical diagnosis. We propose SAM-LAD, a zero-shot, plug-and-play framework for logical anomaly detection in any scene. We validate our proposed SAM-LAD using various benchmarks, including industrial datasets.
• Score: 17.32019706857109
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual anomaly detection is vital in real-world applications, such as industrial defect detection and medical diagnosis. However, most existing methods focus on local structural anomalies and fail to detect higher-level functional anomalies under logical conditions. Although recent studies have explored logical anomaly detection, they can only address simple anomalies like missing or addition and show poor generalizability due to being heavily data-driven. To fill this gap, we propose SAM-LAD, a zero-shot, plug-and-play framework for logical anomaly detection in any scene. First, we obtain a query image's feature map using a pre-trained backbone. Simultaneously, we retrieve the reference images and their corresponding feature maps via the nearest neighbor search of the query image. Then, we introduce the Segment Anything Model (SAM) to obtain object masks of the query and reference images. Each object mask is multiplied with the entire image's feature map to obtain object feature maps. Next, an Object Matching Model (OMM) is proposed to match objects in the query and reference images. To facilitate object matching, we further propose a Dynamic Channel Graph Attention (DCGA) module, treating each object as a keypoint and converting its feature maps into feature vectors. Finally, based on the object matching relations, an Anomaly Measurement Model (AMM) is proposed to detect objects with logical anomalies. Structural anomalies in the objects can also be detected. We validate our proposed SAM-LAD using various benchmarks, including industrial datasets (MVTec Loco AD, MVTec AD), and the logical dataset (DigitAnatomy). Extensive experimental results demonstrate that SAM-LAD outperforms existing SoTA methods, particularly in detecting logical anomalies.

Related papers

• Training-Free Open-Ended Object Detection and Segmentation via Attention as Prompts [14.631774737903015]
  Existing perception models achieve great success by learning from large amounts of labeled data, but they still struggle with open-world scenarios. We present textiti.e., open-ended object detection, which discovers unseen objects without any object categories as inputs. We show that our method surpasses the previous open-ended method on the object detection task and can provide additional instance segmentation masks.
  arXiv  Detail & Related papers  (2024-10-08T12:15:08Z)
• Open-Set Semantic Uncertainty Aware Metric-Semantic Graph Matching [10.439907158831303]
  A metric of semantic uncertainty for open-set object detections is calculated and incorporated into an object-level uncertainty tracking framework. The proposed methods are feasible for real-time use in marine environments for the robust, open-set, multi-object, semantic-uncertainty-aware loop closure detection.
  arXiv  Detail & Related papers  (2024-09-17T20:53:47Z)
• PUAD: Frustratingly Simple Method for Robust Anomaly Detection [0.0]
  We argue that logical anomalies, such as the wrong number of objects, can not be well-represented by the spatial feature maps. We propose a method that incorporates a simple out-of-distribution detection method on the feature space against state-of-the-art reconstruction-based approaches. Our method achieves state-of-the-art performance on the MVTec LOCO AD dataset.
  arXiv  Detail & Related papers  (2024-02-23T06:57:31Z)
• AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection [30.679012320439625]
  AnomalyCLIP learns object-agnostic text prompts to capture generic normality and abnormality in an image. It achieves superior zero-shot performance of detecting and segmenting anomalies in datasets of highly diverse class semantics.
  arXiv  Detail & Related papers  (2023-10-29T10:03:49Z)
• Object-Centric Multiple Object Tracking [124.30650395969126]
  This paper proposes a video object-centric model for multiple-object tracking pipelines. It consists of an index-merge module that adapts the object-centric slots into detection outputs and an object memory module. Benefited from object-centric learning, we only require sparse detection labels for object localization and feature binding.
  arXiv  Detail & Related papers  (2023-09-01T03:34:12Z)
• Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
  Unsupervised object discovery is promising due to its ability to discover objects in a generic manner. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images. We introduce Principal Component Analysis (PCA) to localize object regions.
  arXiv  Detail & Related papers  (2023-07-07T04:03:48Z)
• Self-Supervised Predictive Convolutional Attentive Block for Anomaly Detection [97.93062818228015]
  We propose to integrate the reconstruction-based functionality into a novel self-supervised predictive architectural building block. Our block is equipped with a loss that minimizes the reconstruction error with respect to the masked area in the receptive field. We demonstrate the generality of our block by integrating it into several state-of-the-art frameworks for anomaly detection on image and video.
  arXiv  Detail & Related papers  (2021-11-17T13:30:31Z)
• ODDObjects: A Framework for Multiclass Unsupervised Anomaly Detection on Masked Objects [0.0]
  ODDObjects is designed to detect anomalies of various categories using unsupervised autoencoders trained on COCO-style datasets. The framework extends previous work on anomaly detection with autoencoders, comparing state-of-the-art models trained on object recognition datasets.
  arXiv  Detail & Related papers  (2021-04-26T01:13:28Z)
• Object Detection Made Simpler by Eliminating Heuristic NMS [70.93004137521946]
  We show a simple NMS-free, end-to-end object detection framework. We attain on par or even improved detection accuracy compared with the original one-stage detector.
  arXiv  Detail & Related papers  (2021-01-28T02:38:29Z)
• Slender Object Detection: Diagnoses and Improvements [74.40792217534]
  In this paper, we are concerned with the detection of a particular type of objects with extreme aspect ratios, namely textbfslender objects. For a classical object detection method, a drastic drop of $18.9%$ mAP on COCO is observed, if solely evaluated on slender objects.
  arXiv  Detail & Related papers  (2020-11-17T09:39:42Z)
• Adaptive Object Detection with Dual Multi-Label Prediction [78.69064917947624]
  We propose a novel end-to-end unsupervised deep domain adaptation model for adaptive object detection. The model exploits multi-label prediction to reveal the object category information in each image. We introduce a prediction consistency regularization mechanism to assist object detection.
  arXiv  Detail & Related papers  (2020-03-29T04:23:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.