Towards Robust Vision Transformer via Masked Adaptive Ensemble

• URL: http://arxiv.org/abs/2407.15385v1
• Date: Mon, 22 Jul 2024 05:28:29 GMT
• Title: Towards Robust Vision Transformer via Masked Adaptive Ensemble
• Authors: Fudong Lin, Jiadong Lou, Xu Yuan, Nian-Feng Tzeng,
• Abstract summary: Adversarial training (AT) can help improve the robustness of Vision Transformers (ViT) against adversarial attacks. This paper proposes a novel ViT architecture, including a detector and a classifier bridged by our newly developed adaptive ensemble. Experimental results exhibit that our ViT architecture, on CIFAR-10, achieves the best standard accuracy and adversarial robustness of 90.3% and 49.8%, respectively.
• Score: 23.986968861837813
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Adversarial training (AT) can help improve the robustness of Vision Transformers (ViT) against adversarial attacks by intentionally injecting adversarial examples into the training data. However, this way of adversarial injection inevitably incurs standard accuracy degradation to some extent, thereby calling for a trade-off between standard accuracy and robustness. Besides, the prominent AT solutions are still vulnerable to adaptive attacks. To tackle such shortcomings, this paper proposes a novel ViT architecture, including a detector and a classifier bridged by our newly developed adaptive ensemble. Specifically, we empirically discover that detecting adversarial examples can benefit from the Guided Backpropagation technique. Driven by this discovery, a novel Multi-head Self-Attention (MSA) mechanism is introduced to enhance our detector to sniff adversarial examples. Then, a classifier with two encoders is employed for extracting visual representations respectively from clean images and adversarial examples, with our adaptive ensemble to adaptively adjust the proportion of visual representations from the two encoders for accurate classification. This design enables our ViT architecture to achieve a better trade-off between standard accuracy and robustness. Besides, our adaptive ensemble technique allows us to mask off a random subset of image patches within input data, boosting our ViT's robustness against adaptive attacks, while maintaining high standard accuracy. Experimental results exhibit that our ViT architecture, on CIFAR-10, achieves the best standard accuracy and adversarial robustness of 90.3% and 49.8%, respectively.

Related papers

• Fortify the Guardian, Not the Treasure: Resilient Adversarial Detectors [0.0]
  An adaptive attack is one where the attacker is aware of the defenses and adapts their strategy accordingly. Our proposed method leverages adversarial training to reinforce the ability to detect attacks, without compromising clean accuracy. Experimental evaluations on the CIFAR-10 and SVHN datasets demonstrate that our proposed algorithm significantly improves a detector's ability to accurately identify adaptive adversarial attacks.
  arXiv  Detail & Related papers  (2024-04-18T12:13:09Z)
• Towards Robust Image Stitching: An Adaptive Resistance Learning against Compatible Attacks [66.98297584796391]
  Image stitching seamlessly integrates images captured from varying perspectives into a single wide field-of-view image. Given a pair of captured images, subtle perturbations and distortions which go unnoticed by the human visual system tend to attack the correspondence matching. This paper presents the first attempt to improve the robustness of image stitching against adversarial attacks.
  arXiv  Detail & Related papers  (2024-02-25T02:36:33Z)
• Generalized Face Forgery Detection via Adaptive Learning for Pre-trained Vision Transformer [54.32283739486781]
  We present a textbfForgery-aware textbfAdaptive textbfVision textbfTransformer (FA-ViT) under the adaptive learning paradigm. FA-ViT achieves 93.83% and 78.32% AUC scores on Celeb-DF and DFDC datasets in the cross-dataset evaluation.
  arXiv  Detail & Related papers  (2023-09-20T06:51:11Z)
• Carefully Blending Adversarial Training and Purification Improves Adversarial Robustness [1.2289361708127877]
  CARSO is able to defend itself against adaptive end-to-end white-box attacks devised for defences. Our method improves by a significant margin the state-of-the-art for CIFAR-10, CIFAR-100, and TinyImageNet-200.
  arXiv  Detail & Related papers  (2023-05-25T09:04:31Z)
• Improving Adversarial Robustness to Sensitivity and Invariance Attacks with Deep Metric Learning [80.21709045433096]
  A standard method in adversarial robustness assumes a framework to defend against samples crafted by minimally perturbing a sample. We use metric learning to frame adversarial regularization as an optimal transport problem. Our preliminary results indicate that regularizing over invariant perturbations in our framework improves both invariant and sensitivity defense.
  arXiv  Detail & Related papers  (2022-11-04T13:54:02Z)
• Towards Adversarial Purification using Denoising AutoEncoders [0.8701566919381223]
  Adversarial attacks are often obtained by making subtle perturbations to normal images, which are mostly imperceptible to humans. We propose a framework, named APuDAE, leveraging Denoising AutoEncoders (DAEs) to purify these samples by using them in an adaptive way. We show how our framework provides comparable and in most cases better performance to the baseline methods in purifying adversaries.
  arXiv  Detail & Related papers  (2022-08-29T19:04:25Z)
• Self-Ensembling Vision Transformer (SEViT) for Robust Medical Image Classification [4.843654097048771]
  Vision Transformers (ViT) are competing to replace Convolutional Neural Networks (CNN) for various computer vision tasks in medical imaging. Recent works have shown that ViTs are also susceptible to such attacks and suffer significant performance degradation under attack. We propose a novel self-ensembling method to enhance the robustness of ViT in the presence of adversarial attacks.
  arXiv  Detail & Related papers  (2022-08-04T19:02:24Z)
• Deeper Insights into ViTs Robustness towards Common Corruptions [82.79764218627558]
  We investigate how CNN-like architectural designs and CNN-based data augmentation strategies impact on ViTs' robustness towards common corruptions. We demonstrate that overlapping patch embedding and convolutional Feed-Forward Network (FFN) boost performance on robustness. We also introduce a novel conditional method enabling input-varied augmentations from two angles.
  arXiv  Detail & Related papers  (2022-04-26T08:22:34Z)
• Interpolated Joint Space Adversarial Training for Robust and Generalizable Defenses [82.3052187788609]
  Adversarial training (AT) is considered to be one of the most reliable defenses against adversarial attacks. Recent works show generalization improvement with adversarial samples under novel threat models. We propose a novel threat model called Joint Space Threat Model (JSTM) Under JSTM, we develop novel adversarial attacks and defenses.
  arXiv  Detail & Related papers  (2021-12-12T21:08:14Z)
• Adaptive Feature Alignment for Adversarial Training [56.17654691470554]
  CNNs are typically vulnerable to adversarial attacks, which pose a threat to security-sensitive applications. We propose the adaptive feature alignment (AFA) to generate features of arbitrary attacking strengths. Our method is trained to automatically align features of arbitrary attacking strength.
  arXiv  Detail & Related papers  (2021-05-31T17:01:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.