On the interplay of adversarial robustness and architecture components: patches, convolution and attention

• URL: http://arxiv.org/abs/2209.06953v1
• Date: Wed, 14 Sep 2022 22:02:32 GMT
• Title: On the interplay of adversarial robustness and architecture components: patches, convolution and attention
• Authors: Francesco Croce, Matthias Hein
• Abstract summary: We study the effect of adversarial training on the interpretability of the learnt features and robustness to unseen threat models. An ablation from ResNet to ConvNeXt reveals key architectural changes leading to almost $10%$ higher $ell_infty$-robustness.
• Score: 65.20660287833537
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years novel architecture components for image classification have been developed, starting with attention and patches used in transformers. While prior works have analyzed the influence of some aspects of architecture components on the robustness to adversarial attacks, in particular for vision transformers, the understanding of the main factors is still limited. We compare several (non)-robust classifiers with different architectures and study their properties, including the effect of adversarial training on the interpretability of the learnt features and robustness to unseen threat models. An ablation from ResNet to ConvNeXt reveals key architectural changes leading to almost $10\%$ higher $\ell_\infty$-robustness.

Related papers

• Learning Correlation Structures for Vision Transformers [93.22434535223587]
  We introduce a new attention mechanism, dubbed structural self-attention (StructSA) We generate attention maps by recognizing space-time structures of key-query correlations via convolution. This effectively leverages rich structural patterns in images and videos such as scene layouts, object motion, and inter-object relations.
  arXiv  Detail & Related papers  (2024-04-05T07:13:28Z)
• RBFormer: Improve Adversarial Robustness of Transformer by Robust Bias [18.705151702198854]
  Introducing adversarial examples as a robustness consideration has had a profound and detrimental impact on the performance of well-established convolution-based structures. In this paper, we employ a rational structure design approach to mitigate such vulnerabilities. We introduce a novel structure called Robust Bias Transformer-based Structure (RBFormer) that shows robust superiority compared to several existing baseline structures.
  arXiv  Detail & Related papers  (2023-09-23T03:55:51Z)
• PAIF: Perception-Aware Infrared-Visible Image Fusion for Attack-Tolerant Semantic Segmentation [50.556961575275345]
  We propose a perception-aware fusion framework to promote segmentation robustness in adversarial scenes. We show that our scheme substantially enhances the robustness, with gains of 15.3% mIOU, compared with advanced competitors.
  arXiv  Detail & Related papers  (2023-08-08T01:55:44Z)
• Effects of Architectures on Continual Semantic Segmentation [0.0]
  We study how the choice of neural network architecture affects catastrophic forgetting in class- and domain-incremental semantic segmentation. We find that traditional CNNs like ResNet have high plasticity but low stability, while transformer architectures are much more stable.
  arXiv  Detail & Related papers  (2023-02-21T15:12:01Z)
• Demystify Transformers & Convolutions in Modern Image Deep Networks [82.32018252867277]
  This paper aims to identify the real gains of popular convolution and attention operators through a detailed study. We find that the key difference among these feature transformation modules, such as attention or convolution, lies in their spatial feature aggregation approach. Our experiments on various tasks and an analysis of inductive bias show a significant performance boost due to advanced network-level and block-level designs.
  arXiv  Detail & Related papers  (2022-11-10T18:59:43Z)
• Exploring the Relationship between Architecture and Adversarially Robust Generalization [110.00366382964472]
  Adversarial training is one of the most effective remedies for defending adversarial examples. It often suffers from the huge robustness generalization gap on unseen testing adversaries. This paper tries to bridge the gap by systematically examining the most representative architectures.
  arXiv  Detail & Related papers  (2022-09-28T13:55:28Z)
• Vision Transformer with Convolutions Architecture Search [72.70461709267497]
  We propose an architecture search method-Vision Transformer with Convolutions Architecture Search (VTCAS) The high-performance backbone network searched by VTCAS introduces the desirable features of convolutional neural networks into the Transformer architecture. It enhances the robustness of the neural network for object recognition, especially in the low illumination indoor scene.
  arXiv  Detail & Related papers  (2022-03-20T02:59:51Z)
• Architectural Resilience to Foreground-and-Background Adversarial Noise [0.0]
  Adrial attacks in the form of imperceptible perturbations of normal images have been extensively studied. We propose distinct model-agnostic benchmark perturbations of images to investigate resilience and robustness of different network architectures.
  arXiv  Detail & Related papers  (2020-03-23T01:38:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.