Few-shot Defect Image Generation based on Consistency Modeling

• URL: http://arxiv.org/abs/2408.00372v1
• Date: Thu, 1 Aug 2024 08:29:42 GMT
• Title: Few-shot Defect Image Generation based on Consistency Modeling
• Authors: Qingfeng Shi, Jing Wei, Fei Shen, Zhengtao Zhang,
• Abstract summary: DefectDiffu is a novel text-guided diffusion method to model defect consistency across multiple products. We show that DefectDiffu surpasses state-of-the-art methods in terms of generation quality and diversity.
• Score: 1.8029094254659288
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image generation can solve insufficient labeled data issues in defect detection. Most defect generation methods are only trained on a single product without considering the consistencies among multiple products, leading to poor quality and diversity of generated results. To address these issues, we propose DefectDiffu, a novel text-guided diffusion method to model both intra-product background consistency and inter-product defect consistency across multiple products and modulate the consistency perturbation directions to control product type and defect strength, achieving diversified defect image generation. Firstly, we leverage a text encoder to separately provide consistency prompts for background, defect, and fusion parts of the disentangled integrated architecture, thereby disentangling defects and normal backgrounds. Secondly, we propose the double-free strategy to generate defect images through two-stage perturbation of consistency direction, thereby controlling product type and defect strength by adjusting the perturbation scale. Besides, DefectDiffu can generate defect mask annotations utilizing cross-attention maps from the defect part. Finally, to improve the generation quality of small defects and masks, we propose the adaptive attention-enhance loss to increase the attention to defects. Experimental results demonstrate that DefectDiffu surpasses state-of-the-art methods in terms of generation quality and diversity, thus effectively improving downstream defection performance. Moreover, defect perturbation directions can be transferred among various products to achieve zero-shot defect generation, which is highly beneficial for addressing insufficient data issues. The code are available at https://github.com/FFDD-diffusion/DefectDiffu.

Related papers

• Change-Aware Siamese Network for Surface Defects Segmentation under Complex Background [0.6407952035735353]
  We propose a change-aware Siamese network that solves the defect segmentation in a change detection framework. A novel multi-class balanced contrastive loss is introduced to guide the Transformer-based encoder. The difference presented by a distance map is then skip-connected to the change-aware decoder to assist in the location of both inter-class and out-of-class pixel-wise defects.
  arXiv  Detail & Related papers  (2024-09-01T02:48:11Z)
• StealthDiffusion: Towards Evading Diffusion Forensic Detection through Diffusion Model [62.25424831998405]
  StealthDiffusion is a framework that modifies AI-generated images into high-quality, imperceptible adversarial examples. It is effective in both white-box and black-box settings, transforming AI-generated images into high-quality adversarial forgeries.
  arXiv  Detail & Related papers  (2024-08-11T01:22:29Z)
• Progressive Alignment with VLM-LLM Feature to Augment Defect Classification for the ASE Dataset [7.1083241462091165]
  Traditional defect classification approaches are facing with two barriers. Insufficient training data and unstable data quality. We propose the special dataset, including rich data description recorded on image, for defect classification, but the defect feature is uneasy to learn directly.
  arXiv  Detail & Related papers  (2024-04-08T04:17:27Z)
• DiffMAC: Diffusion Manifold Hallucination Correction for High Generalization Blind Face Restoration [62.44659039265439]
  We propose a Diffusion-Information-Diffusion framework to tackle blind face restoration. DiffMAC achieves high-generalization face restoration in diverse degraded scenes and heterogeneous domains. Results demonstrate the superiority of DiffMAC over state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-03-15T08:44:15Z)
• AnomalyDiffusion: Few-Shot Anomaly Image Generation with Diffusion Model [59.08735812631131]
  Anomaly inspection plays an important role in industrial manufacture. Existing anomaly inspection methods are limited in their performance due to insufficient anomaly data. We propose AnomalyDiffusion, a novel diffusion-based few-shot anomaly generation model.
  arXiv  Detail & Related papers  (2023-12-10T05:13:40Z)
• CINFormer: Transformer network with multi-stage CNN feature injection for surface defect segmentation [73.02218479926469]
  We propose a transformer network with multi-stage CNN feature injection for surface defect segmentation. CINFormer presents a simple yet effective feature integration mechanism that injects the multi-level CNN features of the input image into different stages of the transformer network in the encoder. In addition, CINFormer presents a Top-K self-attention module to focus on tokens with more important information about the defects.
  arXiv  Detail & Related papers  (2023-09-22T06:12:02Z)
• Few-Shot Defect Image Generation via Defect-Aware Feature Manipulation [19.018561017953957]
  We propose the first defect image generation method in the challenging few-shot cases. Our method consists of two training stages. First, we train a data-efficient StyleGAN2 on defect-free images as the backbone. Second, we attach defect-aware residual blocks to the backbone, which learn to produce reasonable defect masks.
  arXiv  Detail & Related papers  (2023-03-04T11:43:08Z)
• Defect Transfer GAN: Diverse Defect Synthesis for Data Augmentation [4.559353193715442]
  Defect Transfer GAN (DT-GAN) learns to represent defect types independent of and across various background products. An empirical study on the MVTec AD and two additional datasets showcase DT-GAN outperforms state-of-the-art image synthesis methods. Results show that the augmented data from DT-GAN provides consistent gains even in the few samples regime.
  arXiv  Detail & Related papers  (2023-02-16T15:35:21Z)
• Improving robustness against common corruptions with frequency biased models [112.65717928060195]
  unseen image corruptions can cause a surprisingly large drop in performance. Image corruption types have different characteristics in the frequency spectrum and would benefit from a targeted type of data augmentation. We propose a new regularization scheme that minimizes the total variation (TV) of convolution feature-maps to increase high-frequency robustness.
  arXiv  Detail & Related papers  (2021-03-30T10:44:50Z)
• Defect-GAN: High-Fidelity Defect Synthesis for Automated Defect Inspection [34.699695525216185]
  Defect-GAN is an automated defect synthesis network that generates realistic and diverse defect samples. It learns through defacement and restoration processes, where the defacement generates defects on normal surface images. It can also mimic variations of defects and offer flexible control over the locations and categories of the generated defects.
  arXiv  Detail & Related papers  (2021-03-28T15:53:34Z)
• Encoding Robustness to Image Style via Adversarial Feature Perturbations [72.81911076841408]
  We adapt adversarial training by directly perturbing feature statistics, rather than image pixels, to produce robust models. Our proposed method, Adversarial Batch Normalization (AdvBN), is a single network layer that generates worst-case feature perturbations during training.
  arXiv  Detail & Related papers  (2020-09-18T17:52:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.