Knowledge Distillation for Anomaly Detection
- URL: http://arxiv.org/abs/2310.06047v1
- Date: Mon, 9 Oct 2023 18:02:38 GMT
- Title: Knowledge Distillation for Anomaly Detection
- Authors: Adrian Alan Pol, Ekaterina Govorkova, Sonja Gronroos, Nadezda
Chernyavskaya, Philip Harris, Maurizio Pierini, Isobel Ojalvo, Peter Elmer
- Abstract summary: We present a novel procedure for compressing an unsupervised anomaly detection model into a supervised deployable one.
We suggest a set of techniques to improve the detection sensitivity.
- Score: 1.864421043173559
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Unsupervised deep learning techniques are widely used to identify anomalous
behaviour. The performance of such methods is a product of the amount of
training data and the model size. However, the size is often a limiting factor
for the deployment on resource-constrained devices. We present a novel
procedure based on knowledge distillation for compressing an unsupervised
anomaly detection model into a supervised deployable one and we suggest a set
of techniques to improve the detection sensitivity. Compressed models perform
comparably to their larger counterparts while significantly reducing the size
and memory footprint.
Related papers
- Scale-Aware Contrastive Reverse Distillation for Unsupervised Medical Anomaly Detection [21.398968141561095]
Unsupervised anomaly detection has garnered significant research attention due to its broad applicability.
Recent methods explore various strategies, including memory banks, normalizing flows, self-supervised learning, and knowledge distillation.
We propose a novel scale-aware contrastive reverse distillation model that addresses two key limitations of existing reverse distillation methods.
arXiv Detail & Related papers (2025-03-18T02:10:20Z) - Unsupervised Anomaly Detection Using Diffusion Trend Analysis [48.19821513256158]
We propose a method to detect anomalies by analysis of reconstruction trend depending on the degree of degradation.
The proposed method is validated on an open dataset for industrial anomaly detection.
arXiv Detail & Related papers (2024-07-12T01:50:07Z) - CL-Flow:Strengthening the Normalizing Flows by Contrastive Learning for
Better Anomaly Detection [1.951082473090397]
We propose a self-supervised anomaly detection approach that combines contrastive learning with 2D-Flow.
Compared to mainstream unsupervised approaches, our self-supervised method demonstrates superior detection accuracy, fewer additional model parameters, and faster inference speed.
Our approach showcases new state-of-the-art results, achieving a performance of 99.6% in image-level AUROC on the MVTecAD dataset and 96.8% in image-level AUROC on the BTAD dataset.
arXiv Detail & Related papers (2023-11-12T10:07:03Z) - On the Effectiveness of Image Manipulation Detection in the Age of
Social Media [9.227950734832447]
manipulation detection algorithms often rely on the manipulated regions being sufficiently'' different from the rest of the non-tampered regions in the image.
We present an in-depth analysis of deep learning-based and learning-free methods, assessing their performance on benchmark datasets.
We propose a novel deep learning-based pre-processing technique that accentuates the anomalies present in manipulated regions.
arXiv Detail & Related papers (2023-04-19T04:05:54Z) - Self-Supervised Training with Autoencoders for Visual Anomaly Detection [61.62861063776813]
We focus on a specific use case in anomaly detection where the distribution of normal samples is supported by a lower-dimensional manifold.
We adapt a self-supervised learning regime that exploits discriminative information during training but focuses on the submanifold of normal examples.
We achieve a new state-of-the-art result on the MVTec AD dataset -- a challenging benchmark for visual anomaly detection in the manufacturing domain.
arXiv Detail & Related papers (2022-06-23T14:16:30Z) - TracInAD: Measuring Influence for Anomaly Detection [0.0]
This paper proposes a novel methodology to flag anomalies based on TracIn.
We test our approach using Variational Autoencoders and show that the average influence of a subsample of training points on a test point can serve as a proxy for abnormality.
arXiv Detail & Related papers (2022-05-03T08:20:15Z) - Object-centric and memory-guided normality reconstruction for video
anomaly detection [56.64792194894702]
This paper addresses anomaly detection problem for videosurveillance.
Due to the inherent rarity and heterogeneity of abnormal events, the problem is viewed as a normality modeling strategy.
Our model learns object-centric normal patterns without seeing anomalous samples during training.
arXiv Detail & Related papers (2022-03-07T19:28:39Z) - Self-Supervised Predictive Convolutional Attentive Block for Anomaly
Detection [97.93062818228015]
We propose to integrate the reconstruction-based functionality into a novel self-supervised predictive architectural building block.
Our block is equipped with a loss that minimizes the reconstruction error with respect to the masked area in the receptive field.
We demonstrate the generality of our block by integrating it into several state-of-the-art frameworks for anomaly detection on image and video.
arXiv Detail & Related papers (2021-11-17T13:30:31Z) - Anomaly Detection via Self-organizing Map [52.542991004752]
Anomaly detection plays a key role in industrial manufacturing for product quality control.
Traditional methods for anomaly detection are rule-based with limited generalization ability.
Recent methods based on supervised deep learning are more powerful but require large-scale annotated datasets for training.
arXiv Detail & Related papers (2021-07-21T06:56:57Z) - Interpreting Rate-Distortion of Variational Autoencoder and Using Model
Uncertainty for Anomaly Detection [5.491655566898372]
We build a scalable machine learning system for unsupervised anomaly detection via representation learning.
We revisit VAE from the perspective of information theory to provide some theoretical foundations on using the reconstruction error.
We show empirically the competitive performance of our approach on benchmark datasets.
arXiv Detail & Related papers (2020-05-05T00:03:48Z) - Unsupervised Anomaly Detection with Adversarial Mirrored AutoEncoders [51.691585766702744]
We propose a variant of Adversarial Autoencoder which uses a mirrored Wasserstein loss in the discriminator to enforce better semantic-level reconstruction.
We put forward an alternative measure of anomaly score to replace the reconstruction-based metric.
Our method outperforms the current state-of-the-art methods for anomaly detection on several OOD detection benchmarks.
arXiv Detail & Related papers (2020-03-24T08:26:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.