Revisiting Transferable Adversarial Images: Systemization, Evaluation, and New Insights

• URL: http://arxiv.org/abs/2310.11850v2
• Date: Tue, 16 Sep 2025 06:15:46 GMT
• Title: Revisiting Transferable Adversarial Images: Systemization, Evaluation, and New Insights
• Authors: Zhengyu Zhao, Hanwei Zhang, Renjue Li, Ronan Sicre, Laurent Amsaleg, Michael Backes, Qi Li, Qian Wang, Chao Shen,
• Abstract summary: Transferable adversarial images raise critical security concerns for computer vision systems in real-world, black-box attack scenarios.<n>In this paper, we systemize transfer attacks into five categories around the general machine learning pipeline.<n>We provide the first comprehensive evaluation, with 23 representative attacks against 11 representative defenses.
• Score: 33.09769747941402
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Transferable adversarial images raise critical security concerns for computer vision systems in real-world, black-box attack scenarios. Although many transfer attacks have been proposed, existing research lacks a systematic and comprehensive evaluation. In this paper, we systemize transfer attacks into five categories around the general machine learning pipeline and provide the first comprehensive evaluation, with 23 representative attacks against 11 representative defenses, including the recent, transfer-oriented defense and the real-world Google Cloud Vision. In particular, we identify two main problems of existing evaluations: (1) for attack transferability, lack of intra-category analyses with fair hyperparameter settings, and (2) for attack stealthiness, lack of diverse measures. Our evaluation results validate that these problems have indeed caused misleading conclusions and missing points, and addressing them leads to new, \textit{consensus-challenging} insights, such as (1) an early attack, DI, even outperforms all similar follow-up ones, (2) the state-of-the-art (white-box) defense, DiffPure, is even vulnerable to (black-box) transfer attacks, and (3) even under the same $L_p$ constraint, different attacks yield dramatically different stealthiness results regarding diverse imperceptibility metrics, finer-grained measures, and a user study. We hope that our analyses will serve as guidance on properly evaluating transferable adversarial images and advance the design of attacks and defenses. Code is available at https://github.com/ZhengyuZhao/TransferAttackEval.

Related papers

• Benchmarking Adversarial Robustness and Adversarial Training Strategies for Object Detection [24.70528833663651]
  Object detection models are critical components of automated systems, such as autonomous vehicles and perception-based robots.<n>Progress in defending these models lags behind classification, hindered by a lack of standardized evaluation.<n>It is nearly impossible to thoroughly compare attack or defense methods, as existing work uses different datasets, inconsistent efficiency metrics, and varied measures of perturbation cost.
  arXiv  Detail & Related papers  (2026-02-18T14:33:58Z)
• Debiased Dual-Invariant Defense for Adversarially Robust Person Re-Identification [52.63017280231648]
  Person re-identification (ReID) is a fundamental task in many real-world applications such as pedestrian trajectory tracking.<n>Person ReID models are highly susceptible to adversarial attacks, where imperceptible perturbations to pedestrian images can cause entirely incorrect predictions.<n>We propose a dual-invariant defense framework composed of two main phases.
  arXiv  Detail & Related papers  (2025-11-13T03:56:40Z)
• Meta Invariance Defense Towards Generalizable Robustness to Unknown Adversarial Attacks [62.036798488144306]
  Current defense mainly focuses on the known attacks, but the adversarial robustness to the unknown attacks is seriously overlooked. We propose an attack-agnostic defense method named Meta Invariance Defense (MID) We show that MID simultaneously achieves robustness to the imperceptible adversarial perturbations in high-level image classification and attack-suppression in low-level robust image regeneration.
  arXiv  Detail & Related papers  (2024-04-04T10:10:38Z)
• Explainable and Transferable Adversarial Attack for ML-Based Network Intrusion Detectors [24.1840740489442]
  Machine learning (ML) has proven to be highly vulnerable to adversarial attacks. White-box and black-box adversarial attacks of NIDS have been explored in several studies. This paper introduces ETA, an Explainable Transfer-based Black-Box Adversarial Attack framework.
  arXiv  Detail & Related papers  (2024-01-19T13:43:09Z)
• Fool the Hydra: Adversarial Attacks against Multi-view Object Detection Systems [3.4673556247932225]
  Adrial patches exemplify the tangible manifestation of the threat posed by adversarial attacks on Machine Learning (ML) models in real-world scenarios. Multiview object systems are able to combine data from multiple views, and reach reliable detection results even in difficult environments. Despite its importance in real-world vision applications, the vulnerability of multiview systems to adversarial patches is not sufficiently investigated.
  arXiv  Detail & Related papers  (2023-11-30T20:11:44Z)
• Towards Good Practices in Evaluating Transfer Adversarial Attacks [23.40245805066479]
  We present the first comprehensive evaluation of transfer attacks, covering 23 representative attacks against 9 defenses on ImageNet. In particular, we propose to categorize existing attacks into five categories, which enables our systematic category-wise analyses. We also pay particular attention to stealthiness, by adopting diverse imperceptibility metrics and looking into new, finer-grained characteristics.
  arXiv  Detail & Related papers  (2022-11-17T14:40:31Z)
• Physical Adversarial Attack meets Computer Vision: A Decade Survey [55.38113802311365]
  This paper presents a comprehensive overview of physical adversarial attacks. We take the first step to systematically evaluate the performance of physical adversarial attacks. Our proposed evaluation metric, hiPAA, comprises six perspectives.
  arXiv  Detail & Related papers  (2022-09-30T01:59:53Z)
• Transferability Ranking of Adversarial Examples [20.41013432717447]
  This paper introduces a ranking strategy that refines the transfer attack process. By leveraging a set of diverse surrogate models, our method can predict transferability of adversarial examples. Using our strategy, we were able to raise the transferability of adversarial examples from a mere 20% - akin to random selection-up to near upper-bound levels.
  arXiv  Detail & Related papers  (2022-08-23T11:25:16Z)
• Zero-Query Transfer Attacks on Context-Aware Object Detectors [95.18656036716972]
  Adversarial attacks perturb images such that a deep neural network produces incorrect classification results. A promising approach to defend against adversarial attacks on natural multi-object scenes is to impose a context-consistency check. We present the first approach for generating context-consistent adversarial attacks that can evade the context-consistency check.
  arXiv  Detail & Related papers  (2022-03-29T04:33:06Z)
• Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning [95.60856995067083]
  This work is among the first to perform adversarial defense for ASV without knowing the specific attack algorithms. We propose to perform adversarial defense from two perspectives: 1) adversarial perturbation purification and 2) adversarial perturbation detection. Experimental results show that our detection module effectively shields the ASV by detecting adversarial samples with an accuracy of around 80%.
  arXiv  Detail & Related papers  (2021-06-01T07:10:54Z)
• Local Black-box Adversarial Attacks: A Query Efficient Approach [64.98246858117476]
  Adrial attacks have threatened the application of deep neural networks in security-sensitive scenarios. We propose a novel framework to perturb the discriminative areas of clean examples only within limited queries in black-box attacks. We conduct extensive experiments to show that our framework can significantly improve the query efficiency during black-box perturbing with a high attack success rate.
  arXiv  Detail & Related papers  (2021-01-04T15:32:16Z)
• A Self-supervised Approach for Adversarial Robustness [105.88250594033053]
  Adversarial examples can cause catastrophic mistakes in Deep Neural Network (DNNs) based vision systems. This paper proposes a self-supervised adversarial training mechanism in the input space. It provides significant robustness against the textbfunseen adversarial attacks.
  arXiv  Detail & Related papers  (2020-06-08T20:42:39Z)
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks [65.20660287833537]
  In this paper we propose two extensions of the PGD-attack overcoming failures due to suboptimal step size and problems of the objective function. We then combine our novel attacks with two complementary existing ones to form a parameter-free, computationally affordable and user-independent ensemble of attacks to test adversarial robustness.
  arXiv  Detail & Related papers  (2020-03-03T18:15:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.