Robustness Feature Adapter for Efficient Adversarial Training

• URL: http://arxiv.org/abs/2508.17680v1
• Date: Mon, 25 Aug 2025 05:23:50 GMT
• Title: Robustness Feature Adapter for Efficient Adversarial Training
• Authors: Quanwei Wu, Jun Guo, Wei Wang, Yi Wang,
• Abstract summary: Adversarial training (AT) with projected gradient descent is the most popular method to improve model robustness under adversarial attacks.<n> computational overheads become prohibitively large when AT is applied to large backbone models.<n>We propose a new adapter-based approach for efficient AT directly in the feature space.
• Score: 6.88216456289286
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Adversarial training (AT) with projected gradient descent is the most popular method to improve model robustness under adversarial attacks. However, computational overheads become prohibitively large when AT is applied to large backbone models. AT is also known to have the issue of robust overfitting. This paper contributes to solving both problems simultaneously towards building more trustworthy foundation models. In particular, we propose a new adapter-based approach for efficient AT directly in the feature space. We show that the proposed adapter-based approach can improve the inner-loop convergence quality by eliminating robust overfitting. As a result, it significantly increases computational efficiency and improves model accuracy by generalizing adversarial robustness to unseen attacks. We demonstrate the effectiveness of the new adapter-based approach in different backbone architectures and in AT at scale.

Related papers

• Adaptive Anomaly Detection in Network Flows with Low-Rank Tensor Decompositions and Deep Unrolling [4.944495309580902]
  Anomaly detection (AD) is increasingly recognized as a key component for ensuring the resilience of future communication systems.<n>This work considers AD in network flows using incomplete measurements.<n>We propose a novel block-successive convex approximation algorithm based on a regularized model-fitting objective.<n>Inspired by Bayesian approaches, we extend the model architecture to perform online adaptation to per-flow and per-time-step statistics.
  arXiv  Detail & Related papers  (2024-09-17T19:59:57Z)
• Enhancing Adversarial Transferability with Adversarial Weight Tuning [50.01825144613307]
  adversarial examples (AEs) mislead the model while appearing benign to human observers.<n>AWT is a data-free tuning method that combines gradient-based and model-based attack methods to enhance the transferability of AEs.
  arXiv  Detail & Related papers  (2024-08-18T13:31:26Z)
• Adaptive Batch Normalization Networks for Adversarial Robustness [33.14617293166724]
  Adversarial Training (AT) has been a standard foundation of modern adversarial defense approaches. We propose adaptive Batch Normalization Network (ABNN), inspired by the recent advances in test-time domain adaptation. ABNN consistently improves adversarial robustness against both digital and physically realizable attacks.
  arXiv  Detail & Related papers  (2024-05-20T00:58:53Z)
• Learn from the Past: A Proxy Guided Adversarial Defense Framework with Self Distillation Regularization [53.04697800214848]
  Adversarial Training (AT) is pivotal in fortifying the robustness of deep learning models. AT methods, relying on direct iterative updates for target model's defense, frequently encounter obstacles such as unstable training and catastrophic overfitting. We present a general proxy guided defense framework, LAST' (bf Learn from the Pbf ast)
  arXiv  Detail & Related papers  (2023-10-19T13:13:41Z)
• AROID: Improving Adversarial Robustness Through Online Instance-Wise Data Augmentation [6.625868719336385]
  Adversarial training (AT) is an effective defense against adversarial examples. Data augmentation (DA) was shown to be effective in mitigating robust overfitting if appropriately designed and optimized for AT. This work proposes a new method to automatically learn online, instance-wise, DA policies to improve robust generalization for AT.
  arXiv  Detail & Related papers  (2023-06-12T15:54:52Z)
• A2: Efficient Automated Attacker for Boosting Adversarial Training [15.37987350655307]
  We propose an efficient automated attacker called A2 to boost Adversarial Training (AT) robustness. A2 generates stronger perturbations with low extra cost and reliably improves the robustness of various AT methods against different attacks.
  arXiv  Detail & Related papers  (2022-10-07T13:28:00Z)
• Fast Adversarial Training with Adaptive Step Size [62.37203478589929]
  We study the phenomenon from the perspective of training instances. We propose a simple but effective method, Adversarial Training with Adaptive Step size (ATAS) ATAS learns an instancewise adaptive step size that is inversely proportional to its gradient norm.
  arXiv  Detail & Related papers  (2022-06-06T08:20:07Z)
• Efficient Few-Shot Object Detection via Knowledge Inheritance [62.36414544915032]
  Few-shot object detection (FSOD) aims at learning a generic detector that can adapt to unseen tasks with scarce training samples. We present an efficient pretrain-transfer framework (PTF) baseline with no computational increment. We also propose an adaptive length re-scaling (ALR) strategy to alleviate the vector length inconsistency between the predicted novel weights and the pretrained base weights.
  arXiv  Detail & Related papers  (2022-03-23T06:24:31Z)
• Adaptive Feature Alignment for Adversarial Training [56.17654691470554]
  CNNs are typically vulnerable to adversarial attacks, which pose a threat to security-sensitive applications. We propose the adaptive feature alignment (AFA) to generate features of arbitrary attacking strengths. Our method is trained to automatically align features of arbitrary attacking strength.
  arXiv  Detail & Related papers  (2021-05-31T17:01:05Z)
• Exploring Model Robustness with Adaptive Networks and Improved Adversarial Training [56.82000424924979]
  We propose a conditional normalization module to adapt networks when conditioned on input samples. Our adaptive networks, once adversarially trained, can outperform their non-adaptive counterparts on both clean validation accuracy and robustness.
  arXiv  Detail & Related papers  (2020-05-30T23:23:56Z)
• Improved Adversarial Training via Learned Optimizer [101.38877975769198]
  We propose a framework to improve the robustness of adversarial training models. By co-training's parameters model's weights, the proposed framework consistently improves robustness and steps adaptively for update directions.
  arXiv  Detail & Related papers  (2020-04-25T20:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.