DiffPAD: Denoising Diffusion-based Adversarial Patch Decontamination

• URL: http://arxiv.org/abs/2410.24006v2
• Date: Thu, 14 Nov 2024 14:58:26 GMT
• Title: DiffPAD: Denoising Diffusion-based Adversarial Patch Decontamination
• Authors: Jia Fu, Xiao Zhang, Sepideh Pashami, Fatemeh Rahimian, Anders Holst,
• Abstract summary: DiffPAD is a novel framework that harnesses the power of diffusion models for adversarial patch decontamination. We show that DiffPAD achieves state-of-the-art adversarial robustness against patch attacks and excels in recovering naturalistic images without patch remnants.
• Score: 5.7254228484416325
• License:
• Abstract: In the ever-evolving adversarial machine learning landscape, developing effective defenses against patch attacks has become a critical challenge, necessitating reliable solutions to safeguard real-world AI systems. Although diffusion models have shown remarkable capacity in image synthesis and have been recently utilized to counter $\ell_p$-norm bounded attacks, their potential in mitigating localized patch attacks remains largely underexplored. In this work, we propose DiffPAD, a novel framework that harnesses the power of diffusion models for adversarial patch decontamination. DiffPAD first performs super-resolution restoration on downsampled input images, then adopts binarization, dynamic thresholding scheme and sliding window for effective localization of adversarial patches. Such a design is inspired by the theoretically derived correlation between patch size and diffusion restoration error that is generalized across diverse patch attack scenarios. Finally, DiffPAD applies inpainting techniques to the original input images with the estimated patch region being masked. By integrating closed-form solutions for super-resolution restoration and image inpainting into the conditional reverse sampling process of a pre-trained diffusion model, DiffPAD obviates the need for text guidance or fine-tuning. Through comprehensive experiments, we demonstrate that DiffPAD not only achieves state-of-the-art adversarial robustness against patch attacks but also excels in recovering naturalistic images without patch remnants. The source code is available at https://github.com/JasonFu1998/DiffPAD.

Related papers

• Real-world Adversarial Defense against Patch Attacks based on Diffusion Model [34.86098237949215]
  This paper introduces DIFFender, a novel DIFfusion-based DeFender framework to counter adversarial patch attacks. At the core of our approach is the discovery of the Adversarial Anomaly Perception (AAP) phenomenon. DIFFender seamlessly integrates the tasks of patch localization and restoration within a unified diffusion model framework.
  arXiv  Detail & Related papers  (2024-09-14T10:38:35Z)
• Pixel Is Not A Barrier: An Effective Evasion Attack for Pixel-Domain Diffusion Models [9.905296922309157]
  Diffusion Models have emerged as powerful generative models for high-quality image synthesis, with many subsequent image editing techniques based on them. Previous works have attempted to safeguard images from diffusion-based editing by adding imperceptible perturbations. Our work proposes a novel attacking framework with a feature representation attack loss that exploits vulnerabilities in denoising UNets and a latent optimization strategy to enhance the naturalness of protected images.
  arXiv  Detail & Related papers  (2024-08-21T17:56:34Z)
• Prompt-Agnostic Adversarial Perturbation for Customized Diffusion Models [27.83772742404565]
  We introduce a Prompt-Agnostic Adversarial Perturbation (PAP) method for customized diffusion models. PAP first models the prompt distribution using a Laplace Approximation, and then produces prompt-agnostic perturbations by maximizing a disturbance expectation. This approach effectively tackles the prompt-agnostic attacks, leading to improved defense stability.
  arXiv  Detail & Related papers  (2024-08-20T06:17:56Z)
• StealthDiffusion: Towards Evading Diffusion Forensic Detection through Diffusion Model [62.25424831998405]
  StealthDiffusion is a framework that modifies AI-generated images into high-quality, imperceptible adversarial examples. It is effective in both white-box and black-box settings, transforming AI-generated images into high-quality adversarial forgeries.
  arXiv  Detail & Related papers  (2024-08-11T01:22:29Z)
• Denoising as Adaptation: Noise-Space Domain Adaptation for Image Restoration [64.84134880709625]
  We show that it is possible to perform domain adaptation via the noise space using diffusion models. In particular, by leveraging the unique property of how auxiliary conditional inputs influence the multi-step denoising process, we derive a meaningful diffusion loss. We present crucial strategies such as channel-shuffling layer and residual-swapping contrastive learning in the diffusion model.
  arXiv  Detail & Related papers  (2024-06-26T17:40:30Z)
• DiffMAC: Diffusion Manifold Hallucination Correction for High Generalization Blind Face Restoration [62.44659039265439]
  We propose a Diffusion-Information-Diffusion framework to tackle blind face restoration. DiffMAC achieves high-generalization face restoration in diverse degraded scenes and heterogeneous domains. Results demonstrate the superiority of DiffMAC over state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-03-15T08:44:15Z)
• Natural Adversarial Patch Generation Method Based on Latent Diffusion Model [2.5879126618627204]
  This paper proposes a novel adversarial patch method called the Latent Diffusion Patch (LDP) It polishes the patches and images through the powerful natural abilities of diffusion models, making them more acceptable to the human visual system. Experimental results, both digital and physical worlds, show that LDPs achieve a visual subjectivity score of 87.3%, while still maintaining effective attack capabilities.
  arXiv  Detail & Related papers  (2023-12-27T04:09:44Z)
• Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Model [61.53213964333474]
  We propose a unified framework Adv-Diffusion that can generate imperceptible adversarial identity perturbations in the latent space but not the raw pixel space. Specifically, we propose the identity-sensitive conditioned diffusion generative model to generate semantic perturbations in the surroundings. The designed adaptive strength-based adversarial perturbation algorithm can ensure both attack transferability and stealthiness.
  arXiv  Detail & Related papers  (2023-12-18T15:25:23Z)
• DiAD: A Diffusion-based Framework for Multi-class Anomaly Detection [55.48770333927732]
  We propose a Difusion-based Anomaly Detection (DiAD) framework for multi-class anomaly detection. It consists of a pixel-space autoencoder, a latent-space Semantic-Guided (SG) network with a connection to the stable diffusion's denoising network, and a feature-space pre-trained feature extractor. Experiments on MVTec-AD and VisA datasets demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2023-12-11T18:38:28Z)
• DiffAttack: Evasion Attacks Against Diffusion-Based Adversarial Purification [63.65630243675792]
  Diffusion-based purification defenses leverage diffusion models to remove crafted perturbations of adversarial examples. Recent studies show that even advanced attacks cannot break such defenses effectively. We propose a unified framework DiffAttack to perform effective and efficient attacks against diffusion-based purification defenses.
  arXiv  Detail & Related papers  (2023-10-27T15:17:50Z)
• DIFFender: Diffusion-Based Adversarial Defense against Patch Attacks [34.86098237949214]
  Adversarial attacks, particularly patch attacks, pose significant threats to the robustness and reliability of deep learning models. This paper introduces DIFFender, a novel defense framework that harnesses the capabilities of a text-guided diffusion model to combat patch attacks. DIFFender integrates dual tasks of patch localization and restoration within a single diffusion model framework.
  arXiv  Detail & Related papers  (2023-06-15T13:33:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.