ADPS: Asymmetric Distillation Post-Segmentation for Image Anomaly Detection

• URL: http://arxiv.org/abs/2210.10495v3
• Date: Mon, 24 Jul 2023 07:43:31 GMT
• Title: ADPS: Asymmetric Distillation Post-Segmentation for Image Anomaly Detection
• Authors: Peng Xing, Hao Tang, Jinhui Tang, Zechao Li
• Abstract summary: Knowledge Distillation-based Anomaly Detection (KDAD) methods rely on the teacher-student paradigm to detect and segment anomalous regions. We propose an innovative approach called Asymmetric Distillation Post-Segmentation (ADPS) Our ADPS employs an asymmetric distillation paradigm that takes distinct forms of the same image as the input of the teacher-student networks. We show that ADPS significantly improves Average Precision (AP) metric by 9% and 20% on the MVTec AD and KolektorSDD2 datasets.
• Score: 75.68023968735523
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Knowledge Distillation-based Anomaly Detection (KDAD) methods rely on the teacher-student paradigm to detect and segment anomalous regions by contrasting the unique features extracted by both networks. However, existing KDAD methods suffer from two main limitations: 1) the student network can effortlessly replicate the teacher network's representations, and 2) the features of the teacher network serve solely as a ``reference standard" and are not fully leveraged. Toward this end, we depart from the established paradigm and instead propose an innovative approach called Asymmetric Distillation Post-Segmentation (ADPS). Our ADPS employs an asymmetric distillation paradigm that takes distinct forms of the same image as the input of the teacher-student networks, driving the student network to learn discriminating representations for anomalous regions. Meanwhile, a customized Weight Mask Block (WMB) is proposed to generate a coarse anomaly localization mask that transfers the distilled knowledge acquired from the asymmetric paradigm to the teacher network. Equipped with WMB, the proposed Post-Segmentation Module (PSM) is able to effectively detect and segment abnormal regions with fine structures and clear boundaries. Experimental results demonstrate that the proposed ADPS outperforms the state-of-the-art methods in detecting and segmenting anomalies. Surprisingly, ADPS significantly improves Average Precision (AP) metric by 9% and 20% on the MVTec AD and KolektorSDD2 datasets, respectively.

Related papers

• Dual-Modeling Decouple Distillation for Unsupervised Anomaly Detection [15.89869857998053]
  Over-generalization of the student network to the teacher network may lead to negligible differences in representation capabilities of anomaly. Existing methods address the possible over-generalization by using differentiated students and teachers from the structural perspective. We propose Dual-Modeling Decouple Distillation (DMDD) for the unsupervised anomaly detection.
  arXiv  Detail & Related papers  (2024-08-07T16:39:16Z)
• DFMSD: Dual Feature Masking Stage-wise Knowledge Distillation for Object Detection [6.371066478190595]
  A novel dual feature-masking heterogeneous distillation framework termed DFMSD is proposed for object detection. A masking enhancement strategy is combined with stage-wise learning to improve feature-masking reconstruction. Experiments for the object detection task demonstrate the promise of our approach.
  arXiv  Detail & Related papers  (2024-07-18T04:19:14Z)
• Structural Teacher-Student Normality Learning for Multi-Class Anomaly Detection and Localization [17.543208086457234]
  We introduce a novel approach known as Structural Teacher-Student Normality Learning (SNL) We evaluate our proposed approach on two anomaly detection datasets, MVTecAD and VisA. Our method surpasses the state-of-the-art distillation-based algorithms by a significant margin of 3.9% and 1.5% on MVTecAD and 1.2% and 2.5% on VisA.
  arXiv  Detail & Related papers  (2024-02-27T00:02:24Z)
• Dual-Student Knowledge Distillation Networks for Unsupervised Anomaly Detection [2.06682776181122]
  Student-teacher networks (S-T) are favored in unsupervised anomaly detection. However, vanilla S-T networks are not stable. We propose a novel dual-student knowledge distillation architecture.
  arXiv  Detail & Related papers  (2024-02-01T09:32:39Z)
• Subject-Independent Deep Architecture for EEG-based Motor Imagery Classification [0.5439020425819]
  Motor EEG (MI) classification based on electroencephalogram (EEG) is a widely-used technique in non-invasive brain-computer interface (BCI) systems. We propose a novel subject-independent semi-supervised deep architecture (SSDA) The proposed SSDA consists of two parts: an unsupervised and a supervised element.
  arXiv  Detail & Related papers  (2024-01-27T23:05:51Z)
• Continual-MAE: Adaptive Distribution Masked Autoencoders for Continual Test-Time Adaptation [49.827306773992376]
  Continual Test-Time Adaptation (CTTA) is proposed to migrate a source pre-trained model to continually changing target distributions. Our proposed method attains state-of-the-art performance in both classification and segmentation CTTA tasks.
  arXiv  Detail & Related papers  (2023-12-19T15:34:52Z)
• Weakly Supervised Semantic Segmentation via Alternative Self-Dual Teaching [82.71578668091914]
  This paper establishes a compact learning framework that embeds the classification and mask-refinement components into a unified deep model. We propose a novel alternative self-dual teaching (ASDT) mechanism to encourage high-quality knowledge interaction.
  arXiv  Detail & Related papers  (2021-12-17T11:56:56Z)
• Semi-supervised Domain Adaptive Structure Learning [72.01544419893628]
  Semi-supervised domain adaptation (SSDA) is a challenging problem requiring methods to overcome both 1) overfitting towards poorly annotated data and 2) distribution shift across domains. We introduce an adaptive structure learning method to regularize the cooperation of SSL and DA.
  arXiv  Detail & Related papers  (2021-12-12T06:11:16Z)
• Deep Semi-supervised Knowledge Distillation for Overlapping Cervical Cell Instance Segmentation [54.49894381464853]
  We propose to leverage both labeled and unlabeled data for instance segmentation with improved accuracy by knowledge distillation. We propose a novel Mask-guided Mean Teacher framework with Perturbation-sensitive Sample Mining. Experiments show that the proposed method improves the performance significantly compared with the supervised method learned from labeled data only.
  arXiv  Detail & Related papers  (2020-07-21T13:27:09Z)
• MetricUNet: Synergistic Image- and Voxel-Level Learning for Precise CT Prostate Segmentation via Online Sampling [66.01558025094333]
  We propose a two-stage framework, with the first stage to quickly localize the prostate region and the second stage to precisely segment the prostate. We introduce a novel online metric learning module through voxel-wise sampling in the multi-task network. Our method can effectively learn more representative voxel-level features compared with the conventional learning methods with cross-entropy or Dice loss.
  arXiv  Detail & Related papers  (2020-05-15T10:37:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.