Boosting Adversarial Transferability via Ensemble Non-Attention

• URL: http://arxiv.org/abs/2511.08937v2
• Date: Fri, 14 Nov 2025 01:48:14 GMT
• Title: Boosting Adversarial Transferability via Ensemble Non-Attention
• Authors: Yipeng Zou, Qin Liu, Jie Wu, Yu Peng, Guo Chen, Hui Zhou, Guanghui Ye,
• Abstract summary: We design a novel ensemble attack, NAMEA, which integrates the gradients from the non-attention areas of ensemble models into the iterative gradient optimization process.<n> NAMEA outperforms AdaEA and SMER, the state-of-the-art ensemble attacks by an average of 15.0% and 9.6%, respectively.
• Score: 12.414747362069457
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Ensemble attacks integrate the outputs of surrogate models with diverse architectures, which can be combined with various gradient-based attacks to improve adversarial transferability. However, previous work shows unsatisfactory attack performance when transferring across heterogeneous model architectures. The main reason is that the gradient update directions of heterogeneous surrogate models differ widely, making it hard to reduce the gradient variance of ensemble models while making the best of individual model. To tackle this challenge, we design a novel ensemble attack, NAMEA, which for the first time integrates the gradients from the non-attention areas of ensemble models into the iterative gradient optimization process. Our design is inspired by the observation that the attention areas of heterogeneous models vary sharply, thus the non-attention areas of ViTs are likely to be the focus of CNNs and vice versa. Therefore, we merge the gradients respectively from the attention and non-attention areas of ensemble models so as to fuse the transfer information of CNNs and ViTs. Specifically, we pioneer a new way of decoupling the gradients of non-attention areas from those of attention areas, while merging gradients by meta-learning. Empirical evaluations on ImageNet dataset indicate that NAMEA outperforms AdaEA and SMER, the state-of-the-art ensemble attacks by an average of 15.0% and 9.6%, respectively. This work is the first attempt to explore the power of ensemble non-attention in boosting cross-architecture transferability, providing new insights into launching ensemble attacks.

Related papers

• ViT-EnsembleAttack: Augmenting Ensemble Models for Stronger Adversarial Transferability in Vision Transformers [12.042884657815845]
  We propose ViT-EnsembleAttack to boost overall generalization of ensemble models and reduce the risk of adversarial overfitting.<n>ViT-EnsembleAttack significantly enhances the adversarial transferability of ensemble-based attacks on ViTs, outperforming existing methods by a substantial margin.
  arXiv  Detail & Related papers  (2025-08-17T14:47:31Z)
• Harmonizing Intra-coherence and Inter-divergence in Ensemble Attacks for Adversarial Transferability [29.69932132116216]
  We propose a novel method Harmonized Ensemble for Adversarial Transferability (HEAT)<n>Heated Ensemble for Adversarial Transferability introduces domain generalization into adversarial example generation for the first time.<n> Experimental results demonstrate that HEAT significantly outperforms existing methods across various datasets and settings.
  arXiv  Detail & Related papers  (2025-05-02T10:17:33Z)
• Boosting Adversarial Transferability with Spatial Adversarial Alignment [56.97809949196889]
  Deep neural networks are vulnerable to adversarial examples that exhibit transferability across various models.<n>We propose a technique that employs an alignment loss and leverages a witness model to fine-tune the surrogate model.<n>Experiments on various architectures on ImageNet show that aligned surrogate models based on SAA can provide higher transferable adversarial examples.
  arXiv  Detail & Related papers  (2025-01-02T02:35:47Z)
• Enhancing Adversarial Transferability with Adversarial Weight Tuning [50.01825144613307]
  adversarial examples (AEs) mislead the model while appearing benign to human observers.<n>AWT is a data-free tuning method that combines gradient-based and model-based attack methods to enhance the transferability of AEs.
  arXiv  Detail & Related papers  (2024-08-18T13:31:26Z)
• Visual Prompt Tuning in Null Space for Continual Learning [51.96411454304625]
  Existing prompt-tuning methods have demonstrated impressive performances in continual learning (CL) This paper aims to learn each task by tuning the prompts in the direction orthogonal to the subspace spanned by previous tasks' features. In practice, an effective null-space-based approximation solution has been proposed to implement the prompt gradient projection.
  arXiv  Detail & Related papers  (2024-06-09T05:57:40Z)
• Ensemble Adversarial Defense via Integration of Multiple Dispersed Low Curvature Models [7.8245455684263545]
  In this work, we aim to enhance ensemble diversity by reducing attack transferability. We identify second-order gradients, which depict the loss curvature, as a key factor in adversarial robustness. We introduce a novel regularizer to train multiple more-diverse low-curvature network models.
  arXiv  Detail & Related papers  (2024-03-25T03:44:36Z)
• An Adaptive Model Ensemble Adversarial Attack for Boosting Adversarial Transferability [26.39964737311377]
  We propose an adaptive ensemble attack, dubbed AdaEA, to adaptively control the fusion of the outputs from each model. We achieve considerable improvement over the existing ensemble attacks on various datasets.
  arXiv  Detail & Related papers  (2023-08-05T15:12:36Z)
• Meta Gradient Adversarial Attack [64.5070788261061]
  This paper proposes a novel architecture called Metaversa Gradient Adrial Attack (MGAA), which is plug-and-play and can be integrated with any existing gradient-based attack method. Specifically, we randomly sample multiple models from a model zoo to compose different tasks and iteratively simulate a white-box attack and a black-box attack in each task. By narrowing the gap between the gradient directions in white-box and black-box attacks, the transferability of adversarial examples on the black-box setting can be improved.
  arXiv  Detail & Related papers  (2021-08-09T17:44:19Z)
• Semantic Correspondence with Transformers [68.37049687360705]
  We propose Cost Aggregation with Transformers (CATs) to find dense correspondences between semantically similar images. We include appearance affinity modelling to disambiguate the initial correlation maps and multi-level aggregation. We conduct experiments to demonstrate the effectiveness of the proposed model over the latest methods and provide extensive ablation studies.
  arXiv  Detail & Related papers  (2021-06-04T14:39:03Z)
• A Transfer Learning approach to Heatmap Regression for Action Unit intensity estimation [50.261472059743845]
  Action Units (AUs) are geometrically-based atomic facial muscle movements. We propose a novel AU modelling problem that consists of jointly estimating their localisation and intensity. A Heatmap models whether an AU occurs or not at a given spatial location.
  arXiv  Detail & Related papers  (2020-04-14T16:51:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.