SeaS: Few-shot Industrial Anomaly Image Generation with Separation and Sharing Fine-tuning

• URL: http://arxiv.org/abs/2410.14987v1
• Date: Sat, 19 Oct 2024 05:37:11 GMT
• Title: SeaS: Few-shot Industrial Anomaly Image Generation with Separation and Sharing Fine-tuning
• Authors: Zhewei Dai, Shilei Zeng, Haotian Liu, Xurui Li, Feng Xue, Yu Zhou,
• Abstract summary: We explore producing diverse anomalies and accurate pixel-wise annotations. By observing the real production lines, we find that anomalies vary randomly in shape and appearance. We produce high-fidelity annotations for the generated anomalies by fusing discriminative features of U-Net and high-resolution VAE features.
• Score: 15.230365669066803
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current segmentation methods require many training images and precise masks, while insufficient anomaly images hinder their application in industrial scenarios. To address such an issue, we explore producing diverse anomalies and accurate pixel-wise annotations. By observing the real production lines, we find that anomalies vary randomly in shape and appearance, whereas products hold globally consistent patterns with slight local variations. Such a characteristic inspires us to develop a Separation and Sharing Fine-tuning (SeaS) approach using only a few abnormal and some normal images. Firstly, we propose the Unbalanced Abnormal (UA) Text Prompt tailored to industrial anomaly generation, consisting of one product token and several anomaly tokens. Then, for anomaly images, we propose a Decoupled Anomaly Alignment (DA) loss to bind the attributes of the anomalies to different anomaly tokens. Re-blending such attributes may produce never-seen anomalies, achieving a high diversity of anomalies. For normal images, we propose a Normal-image Alignment (NA) loss to learn the products' key features that are used to synthesize products with both global consistency and local variations. The two training processes are separated but conducted on a shared U-Net. Finally, SeaS produces high-fidelity annotations for the generated anomalies by fusing discriminative features of U-Net and high-resolution VAE features. Extensive evaluations on the challenging MVTec AD and MVTec 3D AD dataset demonstrate the effectiveness of our approach. For anomaly image generation, we achieve 1.88 on IS and 0.34 on IC-LPIPS on MVTec AD dataset, 1.95 on IS and 0.30 on IC-LPIPS on MVTec 3D AD dataset. For downstream task, using our generated anomaly image-mask pairs, three common segmentation methods achieve an average 11.17% improvement on IoU on MVTec AD dataset, and a 15.49% enhancement in IoU on MVTec 3D AD dataset.

Related papers

• AnoRefiner: Anomaly-Aware Group-Wise Refinement for Zero-Shot Industrial Anomaly Detection [7.619373121202244]
  An anomaly-aware refiner (AnoRefiner) can be plugged into most ZSAD models and improve patch-level anomaly maps to the pixel level.<n>First, we design an anomaly refinement decoder (ARD) that progressively enhances image features using anomaly score maps.<n>Second, motivated by the mass production paradigm, we propose a progressive group-wise test-time training (PGT) strategy.
  arXiv  Detail & Related papers  (2025-11-27T16:25:05Z)
• Unified Unsupervised Anomaly Detection via Matching Cost Filtering [113.43366521994396]
  Unsupervised anomaly detection (UAD) aims to identify image- and pixel-level anomalies using only normal training data.<n>We present Unified Cost Filtering (UCF), a generic post-hoc refinement framework for refining anomaly cost volume of any UAD model.
  arXiv  Detail & Related papers  (2025-10-03T03:28:18Z)
• Generate Aligned Anomaly: Region-Guided Few-Shot Anomaly Image-Mask Pair Synthesis for Industrial Inspection [53.137651284042434]
  Anomaly inspection plays a vital role in industrial manufacturing, but the scarcity of anomaly samples limits the effectiveness of existing methods.<n>We propose Generate grained Anomaly (GAA), a region-guided, few-shot anomaly image-mask pair generation framework.<n>GAA generates realistic, diverse, and semantically aligned anomalies using only a small number of samples.
  arXiv  Detail & Related papers  (2025-07-13T12:56:59Z)
• Learning to Detect Multi-class Anomalies with Just One Normal Image Prompt [4.887838886202545]
  We propose a simple yet effective method that reconstructs normal features and restores anomaly features with just One Normal Image Prompt (OneNIP)<n>In contrast to previous work, OneNIP allows for the first time to reconstruct or restore anomalies with just one normal image prompt, effectively boosting unified anomaly detection performance.<n>OneNIP outperforms previous methods on three industry anomaly detection benchmarks: MVTec, BTAD, and VisA.
  arXiv  Detail & Related papers  (2025-05-14T10:25:14Z)
• AnomalyHybrid: A Domain-agnostic Generative Framework for General Anomaly Detection [3.180143442781838]
  AnomalyHybrid is a domain-agnostic framework designed to generate authentic and diverse anomalies. AnomalyHybrid is a Generative Adversarial Network(GAN)-based framework having two decoders that integrate the appearance of reference image into the depth and edge structures of target image respectively.
  arXiv  Detail & Related papers  (2025-04-06T03:28:30Z)
• KKA: Improving Vision Anomaly Detection through Anomaly-related Knowledge from Large Language Models [54.63075553088399]
  Key Knowledge Augmentation (KKA) is a method that extracts anomaly-related knowledge from large language models (LLMs)<n>KKA classifies the generated anomalies as easy anomalies and hard anomalies according to their similarity to normal samples.<n> Experimental results show that the proposed method significantly improves the performance of various vision anomaly detectors.
  arXiv  Detail & Related papers  (2025-02-14T07:46:49Z)
• GenIAS: Generator for Instantiating Anomalies in time Series [54.959865643340535]
  We develop a generative model for time series anomaly detection (TSAD) using a variational autoencoder.<n>GenIAS is designed to produce diverse and realistic synthetic anomalies for TSAD tasks.<n>Our experiments demonstrate that GenIAS consistently outperforms seventeen traditional and deep anomaly detection models.
  arXiv  Detail & Related papers  (2025-02-12T10:10:04Z)
• Towards Accurate Unified Anomaly Segmentation [25.415671183061317]
  Unsupervised anomaly detection (UAD) from images strives to model normal data distributions, creating discnative representations to distinguish and localize anomalies. Despite recent advancements in the efficient and unified one-for-all scheme, challenges persist in accurately segmenting anomalies for further monitoring. We introduce Unified Anomaly (UniAS) to address the unsolved segmentation task. UniAS achieves state-of-the-art anomaly segmentation performance, attaining 65.12/59.33 and 40.06/32.50 in pAP/DSC on the MVTec-AD and VisA datasets.
  arXiv  Detail & Related papers  (2025-01-21T17:02:51Z)
• AnomalyNCD: Towards Novel Anomaly Class Discovery in Industrial Scenarios [16.77348120041789]
  We propose AnomalyNCD, a multi-class anomaly classification network compatible with different anomaly detection methods. To address the non-prominence of anomalies, we design main element binarization (MEBin) to obtain anomaly-centered images. Next, to learn anomalies with weak semantics, we design mask-guided representation learning, which focuses on isolated anomalies guided by masks.
  arXiv  Detail & Related papers  (2024-10-18T11:07:12Z)
• Fine-grained Abnormality Prompt Learning for Zero-shot Anomaly Detection [88.34095233600719]
  FAPrompt is a novel framework designed to learn Fine-grained Abnormality Prompts for more accurate ZSAD. It substantially outperforms state-of-the-art methods by at least 3%-5% AUC/AP in both image- and pixel-level ZSAD tasks.
  arXiv  Detail & Related papers  (2024-10-14T08:41:31Z)
• Fine-grained Image-to-LiDAR Contrastive Distillation with Visual Foundation Models [55.99654128127689]
  Visual Foundation Models (VFMs) are used to enhance 3D representation learning. VFMs generate semantic labels for weakly-supervised pixel-to-point contrastive distillation. We adapt sampling probabilities of points to address imbalances in spatial distribution and category frequency.
  arXiv  Detail & Related papers  (2024-05-23T07:48:19Z)
• Self-supervised learning for classifying paranasal anomalies in the maxillary sinus [31.45131665942058]
  Self-supervised learning can be used to learn representations from unlabelled data. There are no SSL methods designed for the downstream task of classifying paranasal anomalies in the maxillary sinus. Our approach uses a 3D Convolutional Autoencoder trained in an unsupervised anomaly detection framework.
  arXiv  Detail & Related papers  (2024-04-29T11:14:11Z)
• Self-supervised Feature Adaptation for 3D Industrial Anomaly Detection [59.41026558455904]
  We focus on multi-modal anomaly detection. Specifically, we investigate early multi-modal approaches that attempted to utilize models pre-trained on large-scale visual datasets. We propose a Local-to-global Self-supervised Feature Adaptation (LSFA) method to finetune the adaptors and learn task-oriented representation toward anomaly detection.
  arXiv  Detail & Related papers  (2024-01-06T07:30:41Z)
• DiAD: A Diffusion-based Framework for Multi-class Anomaly Detection [55.48770333927732]
  We propose a Difusion-based Anomaly Detection (DiAD) framework for multi-class anomaly detection. It consists of a pixel-space autoencoder, a latent-space Semantic-Guided (SG) network with a connection to the stable diffusion's denoising network, and a feature-space pre-trained feature extractor. Experiments on MVTec-AD and VisA datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-12-11T18:38:28Z)
• AnomalyDiffusion: Few-Shot Anomaly Image Generation with Diffusion Model [59.08735812631131]
  Anomaly inspection plays an important role in industrial manufacture. Existing anomaly inspection methods are limited in their performance due to insufficient anomaly data. We propose AnomalyDiffusion, a novel diffusion-based few-shot anomaly generation model.
  arXiv  Detail & Related papers  (2023-12-10T05:13:40Z)
• Composite Score for Anomaly Detection in Imbalanced Real-World Industrial Dataset [0.0]
  This paper illustrates a use case for an industrial partner, where Printed Circuit Board Assembly (PCBA) images are reconstructed with a Vector Quantized Generative Adversarial Network (VQGAN) trained on normal products. Several multi-level metrics are extracted on a few normal and abnormal images, highlighting anomalies through reconstruction differences. A classifer is trained to build a composite anomaly score thanks to the metrics extracted.
  arXiv  Detail & Related papers  (2022-11-25T09:41:07Z)
• AnoSeg: Anomaly Segmentation Network Using Self-Supervised Learning [11.234583962952891]
  This paper proposes a novel anomaly segmentation network (AnoSeg) that can directly generate an accurate anomaly map using self-supervised learning. For highly accurate anomaly segmentation, the proposed AnoSeg considers three novel techniques: Anomaly data generation based on hard augmentation, self-supervised learning with pixel-wise and adversarial losses, and coordinate channel concatenation. Our experiments show that the proposed method outperforms the state-of-the-art anomaly detection and anomaly segmentation methods for the MVTec AD dataset.
  arXiv  Detail & Related papers  (2021-10-07T12:36:36Z)
• Explainable Deep Few-shot Anomaly Detection with Deviation Networks [123.46611927225963]
  We introduce a novel weakly-supervised anomaly detection framework to train detection models. The proposed approach learns discriminative normality by leveraging the labeled anomalies and a prior probability. Our model is substantially more sample-efficient and robust, and performs significantly better than state-of-the-art competing methods in both closed-set and open-set settings.
  arXiv  Detail & Related papers  (2021-08-01T14:33:17Z)
• Locally Masked Convolution for Autoregressive Models [107.4635841204146]
  LMConv is a simple modification to the standard 2D convolution that allows arbitrary masks to be applied to the weights at each location in the image. We learn an ensemble of distribution estimators that share parameters but differ in generation order, achieving improved performance on whole-image density estimation.
  arXiv  Detail & Related papers  (2020-06-22T17:59:07Z)
• Modeling the Distribution of Normal Data in Pre-Trained Deep Features for Anomaly Detection [2.9864637081333085]
  Anomaly Detection (AD) in images refers to identifying images and image substructures that deviate significantly from the norm. We show that deep feature representations learned by discriminative models on large natural image datasets are well suited to describe normality.
  arXiv  Detail & Related papers  (2020-05-28T16:43:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.