Neural Network Training Strategy to Enhance Anomaly Detection Performance: A Perspective on Reconstruction Loss Amplification

• URL: http://arxiv.org/abs/2308.14595v1
• Date: Mon, 28 Aug 2023 14:06:36 GMT
• Title: Neural Network Training Strategy to Enhance Anomaly Detection Performance: A Perspective on Reconstruction Loss Amplification
• Authors: YeongHyeon Park, Sungho Kang, Myung Jin Kim, Hyeonho Jeong, Hyunkyu Park, Hyeong Seok Kim, Juneho Yi
• Abstract summary: Unsupervised anomaly detection (UAD) is a widely adopted approach in industry due to rare anomaly occurrences and data imbalance. Recent studies have pursued to contain the generalization capability of their UAD models in reconstruction from different perspectives. We propose a loss landscape sharpening method by amplifying the reconstruction loss, dubbed Loss AMPlification.
• Score: 2.7503452586560484
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unsupervised anomaly detection (UAD) is a widely adopted approach in industry due to rare anomaly occurrences and data imbalance. A desirable characteristic of an UAD model is contained generalization ability which excels in the reconstruction of seen normal patterns but struggles with unseen anomalies. Recent studies have pursued to contain the generalization capability of their UAD models in reconstruction from different perspectives, such as design of neural network (NN) structure and training strategy. In contrast, we note that containing of generalization ability in reconstruction can also be obtained simply from steep-shaped loss landscape. Motivated by this, we propose a loss landscape sharpening method by amplifying the reconstruction loss, dubbed Loss AMPlification (LAMP). LAMP deforms the loss landscape into a steep shape so the reconstruction error on unseen anomalies becomes greater. Accordingly, the anomaly detection performance is improved without any change of the NN architecture. Our findings suggest that LAMP can be easily applied to any reconstruction error metrics in UAD settings where the reconstruction model is trained with anomaly-free samples only.

Related papers

• Structure Is Not Enough: Leveraging Behavior for Neural Network Weight Reconstruction [6.926413609535758]
  One approach to leverage NN weights involves training autoencoders (AEs) using contrastive and reconstruction losses. AEs reconstruct NN models with deteriorated performance compared to the original ones, limiting their usability with regard to model weight generation. We show a strong synergy between structural and behavioral signals, leading to increased performance in all downstream tasks evaluated.
  arXiv  Detail & Related papers  (2025-03-21T13:39:04Z)
• Attention-Guided Perturbation for Unsupervised Image Anomaly Detection [4.084209435209347]
  We present a reconstruction framework named Attention-Guided Perturbation Network (AGPNet) AGPNet learns to add perturbations guided with an attention mask during training. Experiments are conducted on several popular benchmarks covering MVTec-AD, VisA, and MVTec-3D.
  arXiv  Detail & Related papers  (2024-08-14T12:12:43Z)
• SINDER: Repairing the Singular Defects of DINOv2 [61.98878352956125]
  Vision Transformer models trained on large-scale datasets often exhibit artifacts in the patch token they extract. We propose a novel fine-tuning smooth regularization that rectifies structural deficiencies using only a small dataset.
  arXiv  Detail & Related papers  (2024-07-23T20:34:23Z)
• Reconstruction-Based Anomaly Localization via Knowledge-Informed Self-Training [10.565214056914174]
  knowledge-informed self-training (KIST) integrates knowledge into reconstruction model through self-training. KIST utilizes weakly labeled anomalous samples in addition to the normal ones and exploits knowledge to yield pixel-level pseudo-labels of the anomalous samples.
  arXiv  Detail & Related papers  (2024-02-22T03:15:13Z)
• Video Anomaly Detection via Spatio-Temporal Pseudo-Anomaly Generation : A Unified Approach [49.995833831087175]
  This work proposes a novel method for generating generic Video-temporal PAs by inpainting a masked out region of an image. In addition, we present a simple unified framework to detect real-world anomalies under the OCC setting. Our method performs on par with other existing state-of-the-art PAs generation and reconstruction based methods under the OCC setting.
  arXiv  Detail & Related papers  (2023-11-27T13:14:06Z)
• Reconstruction Error-based Anomaly Detection with Few Outlying Examples [1.011824113969195]
  This work investigates approaches to allow reconstruction error-based architectures to instruct the model to put known anomalies outside of the domain description of the normal data. Specifically, our strategy exploits a limited number of anomalous examples to increase the contrast between the reconstruction error associated with normal examples and those associated with both known and unknown anomalies.
  arXiv  Detail & Related papers  (2023-05-17T08:20:29Z)
• DiffusionAD: Norm-guided One-step Denoising Diffusion for Anomaly Detection [89.49600182243306]
  We reformulate the reconstruction process using a diffusion model into a noise-to-norm paradigm. We propose a rapid one-step denoising paradigm, significantly faster than the traditional iterative denoising in diffusion models. The segmentation sub-network predicts pixel-level anomaly scores using the input image and its anomaly-free restoration.
  arXiv  Detail & Related papers  (2023-03-15T16:14:06Z)
• Diversity-Measurable Anomaly Detection [106.07413438216416]
  We propose Diversity-Measurable Anomaly Detection (DMAD) framework to enhance reconstruction diversity. PDM essentially decouples deformation from embedding and makes the final anomaly score more reliable.
  arXiv  Detail & Related papers  (2023-03-09T05:52:42Z)
• Two-stream Decoder Feature Normality Estimating Network for Industrial Anomaly Detection [4.772323272202286]
  We propose a two-stream decoder network (TSDN) to learn both normal and abnormal features. We also propose a feature normality estimator (FNE) to eliminate abnormal features and prevent high-quality reconstruction of abnormal regions.
  arXiv  Detail & Related papers  (2023-02-20T06:46:09Z)
• Prototypical Residual Networks for Anomaly Detection and Localization [80.5730594002466]
  We propose a framework called Prototypical Residual Network (PRN) PRN learns feature residuals of varying scales and sizes between anomalous and normal patterns to accurately reconstruct the segmentation maps of anomalous regions. We present a variety of anomaly generation strategies that consider both seen and unseen appearance variance to enlarge and diversify anomalies.
  arXiv  Detail & Related papers  (2022-12-05T05:03:46Z)
• Generalizable Industrial Visual Anomaly Detection with Self-Induction Vision Transformer [5.116033262865781]
  We propose a self-induction vision Transformer (SIVT) for unsupervised generalizable industrial visual anomaly detection and localization. The proposed SIVT first extracts discriminatory features from pre-trained CNN as property descriptors, then reconstructs the extracted features in a self-supervisory fashion. The results reveal that the proposed method can advance state-of-the-art detection performance with an improvement of 2.8-6.3 in AUROC, and 3.3-7.6 in AP.
  arXiv  Detail & Related papers  (2022-11-22T14:56:12Z)
• Self-Supervised Training with Autoencoders for Visual Anomaly Detection [61.62861063776813]
  We focus on a specific use case in anomaly detection where the distribution of normal samples is supported by a lower-dimensional manifold. We adapt a self-supervised learning regime that exploits discriminative information during training but focuses on the submanifold of normal examples. We achieve a new state-of-the-art result on the MVTec AD dataset -- a challenging benchmark for visual anomaly detection in the manufacturing domain.
  arXiv  Detail & Related papers  (2022-06-23T14:16:30Z)
• Divide-and-Assemble: Learning Block-wise Memory for Unsupervised Anomaly Detection [40.778313918994996]
  Reconstruction-based methods play an important role in unsupervised anomaly detection in images. In this work, we interpret the reconstruction of an image as a divide-and-assemble procedure. We achieve state-of-the-art performance on the challenging MVTec AD dataset.
  arXiv  Detail & Related papers  (2021-07-28T01:14:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.