A Survey of Adversarial Efficiency Degradation for Vision Transformer by Exploiting Input-adaptive Optimization

• URL: http://arxiv.org/abs/2608.05217v1
• Date: Wed, 05 Aug 2026 10:29:06 GMT
• Title: A Survey of Adversarial Efficiency Degradation for Vision Transformer by Exploiting Input-adaptive Optimization
• Abstract summary: Vision Transformers (ViTs) increasingly rely on input-adaptive inference, such as token pruning and early halting, to meet energy and latency budgets.<n>This survey examines a recent class of adversarial efficiency degradation attacks that target these mechanisms to increase without necessarily degrading accuracy.
• Score: 2.458848206334729
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision Transformers (ViTs) increasingly rely on input-adaptive inference, such as token pruning and early halting, to meet energy and latency budgets. This survey examines a recent class of adversarial efficiency degradation attacks that target these mechanisms to increase computation without necessarily degrading accuracy. We unify and compare two representative attacks, SlowFormer (a universal adversarial patch) and DeSparsify (per-image perturbations), across three popular token-pruning frameworks: A-ViT, ATS, and AdaViT. We standardize reporting using GFLOPs, accuracy loss, and an Attack Success (AS) metric that measures how much of the model's compute savings the attack takes away. Understanding these attacks is crucial for designing countermeasures that not only mitigate risk but also remain lightweight, since deployment often occurs in low-power settings such as mobile or embedded devices. To organize our analysis, we focus on three questions: how input-adaptive optimizations (e.g., token pruning and early halting) create attack surfaces for efficiency degradation; how such attacks operate in practice and which optimizations are most vulnerable; and which defenses exist today and whether they meaningfully restore efficiency under attack.

Related papers

• MOAT: Model-Agnostic Randomized Transformations for preventing Efficiency Degradation Attacks on ViTs [5.317090645596605]
  token pruning is widely used to reduce computational cost without impacting accuracy.<n> adversaries have developed targeted attacks against said token pruning techniques to undermine such attempts to make ViTs efficient.<n>We propose MOAT, a model-agnostic pre-processing defense pipeline that applies a combination of input transformations to protect efficient ViT implementations.
  arXiv  Detail & Related papers  (2026-08-05T10:47:58Z)
• ATAAT: Adaptive Threat-Aware Adversarial Tuning Framework against Backdoor Attacks on Vision-Language-Action Models [11.93174278125059]
  This study investigates backdoor attacks targeting the visual pathway.<n>We identify a core obstacle causing the failure of traditional attack paradigms: "Gradient Interference"<n>We propose an Adaptive Threat-Aware Adrial Tuning framework.
  arXiv  Detail & Related papers  (2026-05-09T02:15:10Z)
• EAGER: Edge-Aligned LLM Defense for Robust, Efficient, and Accurate Cybersecurity Question Answering [10.78145758065258]
  EAGER integrates parameter-efficient quantization with domain-specific preference alignment to jointly optimize efficiency, robustness, and accuracy.<n> Experiments show that EAGER reduces adversarial attack success rates by up to 7.3x and improves QA accuracy by up to 55% over state-of-the-art defenses.
  arXiv  Detail & Related papers  (2025-11-24T06:49:48Z)
• Sampling-aware Adversarial Attacks Against Large Language Models [52.30089653615172]
  Existing adversarial attacks typically target harmful responses in single-point greedy generations.<n>We show that for the goal of eliciting harmful responses, repeated sampling of model outputs during the attack prompt optimization.<n>We show that integrating sampling into existing attacks boosts success rates by up to 37% and improves efficiency by up to two orders of magnitude.
  arXiv  Detail & Related papers  (2025-07-06T16:13:33Z)
• Improving Black-Box Generative Attacks via Generator Semantic Consistency [51.470649503929344]
  generative attacks produce adversarial examples in a single forward pass at test time.<n>We enforce semantic consistency by aligning the early generator's intermediate features to an EMA teacher.<n>Our approach can be seamlessly integrated into existing generative attacks with consistent improvements in black-box transfer.
  arXiv  Detail & Related papers  (2025-06-23T02:35:09Z)
• FaultGuard: A Generative Approach to Resilient Fault Prediction in Smart Electrical Grids [53.2306792009435]
  FaultGuard is the first framework for fault type and zone classification resilient to adversarial attacks. We propose a low-complexity fault prediction model and an online adversarial training technique to enhance robustness. Our model outclasses the state-of-the-art for resilient fault prediction benchmarking, with an accuracy of up to 0.958.
  arXiv  Detail & Related papers  (2024-03-26T08:51:23Z)
• DeSparsify: Adversarial Attack Against Token Sparsification Mechanisms in Vision Transformers [25.560501078099826]
  We present DeSparsify, an attack targeting the availability of vision transformers that use token sparsification mechanisms. The attack aims to exhaust the operating system's resources, while maintaining its stealthiness. To mitigate the impact of the attack, we propose various countermeasures.
  arXiv  Detail & Related papers  (2024-02-04T15:59:35Z)
• Revisiting DeepFool: generalization and improvement [16.554225382392993]
  We introduce a new family of adversarial attacks that strike a balance between effectiveness and computational efficiency.<n>Our proposed attacks are also suitable for evaluating the robustness of large models and can be used to perform adversarial training.
  arXiv  Detail & Related papers  (2023-03-22T11:49:35Z)
• Versatile Weight Attack via Flipping Limited Bits [68.45224286690932]
  We study a novel attack paradigm, which modifies model parameters in the deployment stage. Considering the effectiveness and stealthiness goals, we provide a general formulation to perform the bit-flip based weight attack. We present two cases of the general formulation with different malicious purposes, i.e., single sample attack (SSA) and triggered samples attack (TSA)
  arXiv  Detail & Related papers  (2022-07-25T03:24:58Z)
• Evaluating the Robustness of Semantic Segmentation for Autonomous Driving against Real-World Adversarial Patch Attacks [62.87459235819762]
  In a real-world scenario like autonomous driving, more attention should be devoted to real-world adversarial examples (RWAEs) This paper presents an in-depth evaluation of the robustness of popular SS models by testing the effects of both digital and real-world adversarial patches.
  arXiv  Detail & Related papers  (2021-08-13T11:49:09Z)
• Adaptive Feature Alignment for Adversarial Training [56.17654691470554]
  CNNs are typically vulnerable to adversarial attacks, which pose a threat to security-sensitive applications. We propose the adaptive feature alignment (AFA) to generate features of arbitrary attacking strengths. Our method is trained to automatically align features of arbitrary attacking strength.
  arXiv  Detail & Related papers  (2021-05-31T17:01:05Z)
• A Self-supervised Approach for Adversarial Robustness [105.88250594033053]
  Adversarial examples can cause catastrophic mistakes in Deep Neural Network (DNNs) based vision systems. This paper proposes a self-supervised adversarial training mechanism in the input space. It provides significant robustness against the textbfunseen adversarial attacks.
  arXiv  Detail & Related papers  (2020-06-08T20:42:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.