Beyond Pretrained Features: Noisy Image Modeling Provides Adversarial Defense

• URL: http://arxiv.org/abs/2302.01056v3
• Date: Thu, 9 Nov 2023 12:21:02 GMT
• Title: Beyond Pretrained Features: Noisy Image Modeling Provides Adversarial Defense
• Authors: Zunzhi You, Daochang Liu, Bohyung Han, Chang Xu
• Abstract summary: Masked image modeling (MIM) has made it a prevailing framework for self-supervised visual representation learning. In this paper, we investigate how this powerful self-supervised learning paradigm can provide adversarial robustness to downstream classifiers. We propose an adversarial defense method, referred to as De3, by exploiting the pretrained decoder for denoising.
• Score: 52.66971714830943
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advancements in masked image modeling (MIM) have made it a prevailing framework for self-supervised visual representation learning. The MIM pretrained models, like most deep neural network methods, remain vulnerable to adversarial attacks, limiting their practical application, and this issue has received little research attention. In this paper, we investigate how this powerful self-supervised learning paradigm can provide adversarial robustness to downstream classifiers. During the exploration, we find that noisy image modeling (NIM), a simple variant of MIM that adopts denoising as the pre-text task, reconstructs noisy images surprisingly well despite severe corruption. Motivated by this observation, we propose an adversarial defense method, referred to as De^3, by exploiting the pretrained decoder for denoising. Through De^3, NIM is able to enhance adversarial robustness beyond providing pretrained features. Furthermore, we incorporate a simple modification, sampling the noise scale hyperparameter from random distributions, and enable the defense to achieve a better and tunable trade-off between accuracy and robustness. Experimental results demonstrate that, in terms of adversarial robustness, NIM is superior to MIM thanks to its effective denoising capability. Moreover, the defense provided by NIM achieves performance on par with adversarial training while offering the extra tunability advantage. Source code and models are available at https://github.com/youzunzhi/NIM-AdvDef.

Related papers

• Protecting Feed-Forward Networks from Adversarial Attacks Using Predictive Coding [0.20718016474717196]
  An adversarial example is a modified input image designed to cause a Machine Learning (ML) model to make a mistake. This study presents a practical and effective solution -- using predictive coding networks (PCnets) as an auxiliary step for adversarial defence.
  arXiv  Detail & Related papers  (2024-10-31T21:38:05Z)
• Adversarial Robustification via Text-to-Image Diffusion Models [56.37291240867549]
  Adrial robustness has been conventionally believed as a challenging property to encode for neural networks. We develop a scalable and model-agnostic solution to achieve adversarial robustness without using any data.
  arXiv  Detail & Related papers  (2024-07-26T10:49:14Z)
• Advancing the Robustness of Large Language Models through Self-Denoised Smoothing [50.54276872204319]
  Large language models (LLMs) have achieved significant success, but their vulnerability to adversarial perturbations has raised considerable concerns. We propose to leverage the multitasking nature of LLMs to first denoise the noisy inputs and then to make predictions based on these denoised versions. Unlike previous denoised smoothing techniques in computer vision, which require training a separate model to enhance the robustness of LLMs, our method offers significantly better efficiency and flexibility.
  arXiv  Detail & Related papers  (2024-04-18T15:47:00Z)
• Perturbation-Invariant Adversarial Training for Neural Ranking Models: Improving the Effectiveness-Robustness Trade-Off [107.35833747750446]
  adversarial examples can be crafted by adding imperceptible perturbations to legitimate documents. This vulnerability raises significant concerns about their reliability and hinders the widespread deployment of NRMs. In this study, we establish theoretical guarantees regarding the effectiveness-robustness trade-off in NRMs.
  arXiv  Detail & Related papers  (2023-12-16T05:38:39Z)
• Stable Unlearnable Example: Enhancing the Robustness of Unlearnable Examples via Stable Error-Minimizing Noise [31.586389548657205]
  Unlearnable example is proposed to significantly degrade the generalization performance of models by adding a kind of imperceptible noise to the data. We introduce stable error-minimizing noise (SEM), which trains the defensive noise against random perturbation instead of the time-consuming adversarial perturbation. SEM achieves a new state-of-the-art performance on CIFAR-10, CIFAR-100, and ImageNet Subset.
  arXiv  Detail & Related papers  (2023-11-22T01:43:57Z)
• Evaluating Similitude and Robustness of Deep Image Denoising Models via Adversarial Attack [60.40356882897116]
  Deep neural networks (DNNs) have shown superior performance compared to traditional image denoising algorithms. In this paper, we propose an adversarial attack method named denoising-PGD which can successfully attack all the current deep denoising models.
  arXiv  Detail & Related papers  (2023-06-28T09:30:59Z)
• Wavelets Beat Monkeys at Adversarial Robustness [0.8702432681310401]
  We show how physically inspired structures yield new insights into robustness that were previously only thought possible by meticulously mimicking the human cortex. Our work shows how physically inspired structures yield new insights into robustness that were previously only thought possible by meticulously mimicking the human cortex.
  arXiv  Detail & Related papers  (2023-04-19T03:41:30Z)
• DIMBA: Discretely Masked Black-Box Attack in Single Object Tracking [5.672132510411465]
  adversarial attack can force a CNN-based model to produce an incorrect output by craftily manipulating human-imperceptible input. We propose a novel adversarial attack method to generate noises for single object tracking under black-box settings. Our method requires fewer queries on frames of a video to manipulate competitive or even better attack performance.
  arXiv  Detail & Related papers  (2022-07-17T00:17:40Z)
• Stylized Adversarial Defense [105.88250594033053]
  adversarial training creates perturbation patterns and includes them in the training set to robustify the model. We propose to exploit additional information from the feature space to craft stronger adversaries. Our adversarial training approach demonstrates strong robustness compared to state-of-the-art defenses.
  arXiv  Detail & Related papers  (2020-07-29T08:38:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.