AdaGAT: Adaptive Guidance Adversarial Training for the Robustness of Deep Neural Networks

• URL: http://arxiv.org/abs/2508.17265v1
• Date: Sun, 24 Aug 2025 09:11:28 GMT
• Title: AdaGAT: Adaptive Guidance Adversarial Training for the Robustness of Deep Neural Networks
• Abstract summary: We propose a novel Adaptive Guidance Adversarial Training (AdaGAT) method.<n>We develop two separate loss functions as part of the AdaGAT method, allowing the guide model to participate more actively in backpropagation to achieve its optimal state.<n>Our observations reveal that appropriately adjusting the guide model within a certain accuracy range enhances the target model's robustness across various adversarial attacks.
• Score: 5.446956311538973
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Adversarial distillation (AD) is a knowledge distillation technique that facilitates the transfer of robustness from teacher deep neural network (DNN) models to lightweight target (student) DNN models, enabling the target models to perform better than only training the student model independently. Some previous works focus on using a small, learnable teacher (guide) model to improve the robustness of a student model. Since a learnable guide model starts learning from scratch, maintaining its optimal state for effective knowledge transfer during co-training is challenging. Therefore, we propose a novel Adaptive Guidance Adversarial Training (AdaGAT) method. Our method, AdaGAT, dynamically adjusts the training state of the guide model to install robustness to the target model. Specifically, we develop two separate loss functions as part of the AdaGAT method, allowing the guide model to participate more actively in backpropagation to achieve its optimal state. We evaluated our approach via extensive experiments on three datasets: CIFAR-10, CIFAR-100, and TinyImageNet, using the WideResNet-34-10 model as the target model. Our observations reveal that appropriately adjusting the guide model within a certain accuracy range enhances the target model's robustness across various adversarial attacks compared to a variety of baseline models.

Related papers

• Approach to Finding a Robust Deep Learning Model [0.28675177318965045]
  The rapid development of machine learning (ML) and artificial intelligence (AI) applications requires the training of large numbers of models.<n>We propose a novel approach for determining model robustness using a proposed model selection algorithm designed as a meta-algorithm.<n>Within this framework, we address the influence of training sample size, model weight, and inductive bias on the robustness of deep learning models.
  arXiv  Detail & Related papers  (2025-05-22T20:05:20Z)
• Exploring and Enhancing the Transfer of Distribution in Knowledge Distillation for Autoregressive Language Models [62.5501109475725]
  Knowledge distillation (KD) is a technique that compresses large teacher models by training smaller student models to mimic them. This paper introduces Online Knowledge Distillation (OKD), where the teacher network integrates small online modules to concurrently train with the student model. OKD achieves or exceeds the performance of leading methods in various model architectures and sizes, reducing training time by up to fourfold.
  arXiv  Detail & Related papers  (2024-09-19T07:05:26Z)
• Dynamic Label Adversarial Training for Deep Learning Robustness Against Adversarial Attacks [11.389689242531327]
  Adversarial training is one of the most effective methods for enhancing model robustness. Previous approaches primarily use static ground truth for adversarial training, but this often causes robust overfitting. We propose a dynamic label adversarial training (DYNAT) algorithm that enables the target model to gain robustness from the guide model's decisions.
  arXiv  Detail & Related papers  (2024-08-23T14:25:12Z)
• Learn from the Past: A Proxy Guided Adversarial Defense Framework with Self Distillation Regularization [53.04697800214848]
  Adversarial Training (AT) is pivotal in fortifying the robustness of deep learning models. AT methods, relying on direct iterative updates for target model's defense, frequently encounter obstacles such as unstable training and catastrophic overfitting. We present a general proxy guided defense framework, LAST' (bf Learn from the Pbf ast)
  arXiv  Detail & Related papers  (2023-10-19T13:13:41Z)
• Towards Efficient Task-Driven Model Reprogramming with Foundation Models [52.411508216448716]
  Vision foundation models exhibit impressive power, benefiting from the extremely large model capacity and broad training data. However, in practice, downstream scenarios may only support a small model due to the limited computational resources or efficiency considerations. This brings a critical challenge for the real-world application of foundation models: one has to transfer the knowledge of a foundation model to the downstream task.
  arXiv  Detail & Related papers  (2023-04-05T07:28:33Z)
• Controlled Sparsity via Constrained Optimization or: How I Learned to Stop Tuning Penalties and Love Constraints [81.46143788046892]
  We focus on the task of controlling the level of sparsity when performing sparse learning. Existing methods based on sparsity-inducing penalties involve expensive trial-and-error tuning of the penalty factor. We propose a constrained formulation where sparsification is guided by the training objective and the desired sparsity target in an end-to-end fashion.
  arXiv  Detail & Related papers  (2022-08-08T21:24:20Z)
• Online Dynamics Learning for Predictive Control with an Application to Aerial Robots [3.673994921516517]
  Even though prediction models can be learned and applied to model-based controllers, these models are often learned offline. In this offline setting, training data is first collected and a prediction model is learned through an elaborated training procedure. We propose an online dynamics learning framework that continually improves the accuracy of the dynamic model during deployment.
  arXiv  Detail & Related papers  (2022-07-19T15:51:25Z)
• DST: Dynamic Substitute Training for Data-free Black-box Attack [79.61601742693713]
  We propose a novel dynamic substitute training attack method to encourage substitute model to learn better and faster from the target model. We introduce a task-driven graph-based structure information learning constrain to improve the quality of generated training data.
  arXiv  Detail & Related papers  (2022-04-03T02:29:11Z)
• Ensemble Knowledge Distillation for CTR Prediction [46.92149090885551]
  We propose a new model training strategy based on knowledge distillation (KD) KD is a teacher-student learning framework to transfer knowledge learned from a teacher model to a student model. We propose some novel techniques to facilitate ensembled CTR prediction, including teacher gating and early stopping by distillation loss.
  arXiv  Detail & Related papers  (2020-11-08T23:37:58Z)
• Adversarial Concurrent Training: Optimizing Robustness and Accuracy Trade-off of Deep Neural Networks [13.041607703862724]
  We propose Adversarial Concurrent Training (ACT) to train a robust model in conjunction with a natural model in a minimax game. ACT achieves 68.20% standard accuracy and 44.29% robustness accuracy under a 100-iteration untargeted attack.
  arXiv  Detail & Related papers  (2020-08-16T22:14:48Z)
• Improved Adversarial Training via Learned Optimizer [101.38877975769198]
  We propose a framework to improve the robustness of adversarial training models. By co-training's parameters model's weights, the proposed framework consistently improves robustness and steps adaptively for update directions.
  arXiv  Detail & Related papers  (2020-04-25T20:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.