D2R: dual regularization loss with collaborative adversarial generation for model robustness

• URL: http://arxiv.org/abs/2506.07056v1
• Date: Sun, 08 Jun 2025 09:39:54 GMT
• Title: D2R: dual regularization loss with collaborative adversarial generation for model robustness
• Authors: Zhenyu Liu, Huizhi Liang, Rajiv Ranjan, Zhanxing Zhu, Vaclav Snasel, Varun Ojha,
• Abstract summary: robustness of Deep Neural Network models is crucial for defending models against adversarial attacks.<n>We propose a dual regularization loss (D2R Loss) method and a collaborative adversarial generation (CAG) strategy for adversarial training.<n>Our results show that D2R loss with CAG produces highly robust models.
• Score: 23.712462151414726
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The robustness of Deep Neural Network models is crucial for defending models against adversarial attacks. Recent defense methods have employed collaborative learning frameworks to enhance model robustness. Two key limitations of existing methods are (i) insufficient guidance of the target model via loss functions and (ii) non-collaborative adversarial generation. We, therefore, propose a dual regularization loss (D2R Loss) method and a collaborative adversarial generation (CAG) strategy for adversarial training. D2R loss includes two optimization steps. The adversarial distribution and clean distribution optimizations enhance the target model's robustness by leveraging the strengths of different loss functions obtained via a suitable function space exploration to focus more precisely on the target model's distribution. CAG generates adversarial samples using a gradient-based collaboration between guidance and target models. We conducted extensive experiments on three benchmark databases, including CIFAR-10, CIFAR-100, Tiny ImageNet, and two popular target models, WideResNet34-10 and PreActResNet18. Our results show that D2R loss with CAG produces highly robust models.

Related papers

• Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees [3.6787328174619254]
  Two-stage Learning-to-Defer (L2D) enables optimal task delegation by assigning each input to either a fixed main model or one of several offline experts.<n>Existing L2D frameworks assume clean inputs and are vulnerable to adversarial perturbations that can manipulate query allocation.<n>We present the first comprehensive study of adversarial robustness in two-stage L2D systems.
  arXiv  Detail & Related papers  (2025-02-03T03:44:35Z)
• Weak-to-Strong Diffusion with Reflection [56.39451539396458]
  We propose Weak-to-Strong Diffusion (W2SD) to bridge the gap between an ideal model and a strong model.<n>W2SD steers latent variables along sampling trajectories toward regions of the real data distribution.<n>Extensive experiments demonstrate that W2SD significantly improves human preference, aesthetic quality, and prompt adherence.
  arXiv  Detail & Related papers  (2025-02-01T16:00:08Z)
• Towards Adversarially Robust Deep Metric Learning [0.8702432681310401]
  Deep neural networks are prone to adversarial attacks and could be easily fooled by adversarial examples.<n>Existing works fail to thoroughly inspect the robustness of DML models.<n>We propose a new defense, the Ensemble Adversarial Training (EAT), which exploits ensemble learning and adversarial training.
  arXiv  Detail & Related papers  (2025-01-02T03:15:25Z)
• Dynamic Label Adversarial Training for Deep Learning Robustness Against Adversarial Attacks [11.389689242531327]
  Adversarial training is one of the most effective methods for enhancing model robustness. Previous approaches primarily use static ground truth for adversarial training, but this often causes robust overfitting. We propose a dynamic label adversarial training (DYNAT) algorithm that enables the target model to gain robustness from the guide model's decisions.
  arXiv  Detail & Related papers  (2024-08-23T14:25:12Z)
• Model Inversion Attacks Through Target-Specific Conditional Diffusion Models [54.69008212790426]
  Model inversion attacks (MIAs) aim to reconstruct private images from a target classifier's training set, thereby raising privacy concerns in AI applications. Previous GAN-based MIAs tend to suffer from inferior generative fidelity due to GAN's inherent flaws and biased optimization within latent space. We propose Diffusion-based Model Inversion (Diff-MI) attacks to alleviate these issues.
  arXiv  Detail & Related papers  (2024-07-16T06:38:49Z)
• Efficient Adversarial Training in LLMs with Continuous Attacks [99.5882845458567]
  Large language models (LLMs) are vulnerable to adversarial attacks that can bypass their safety guardrails. We propose a fast adversarial training algorithm (C-AdvUL) composed of two losses. C-AdvIPO is an adversarial variant of IPO that does not require utility data for adversarially robust alignment.
  arXiv  Detail & Related papers  (2024-05-24T14:20:09Z)
• Adversarial Fine-tuning of Compressed Neural Networks for Joint Improvement of Robustness and Efficiency [3.3490724063380215]
  Adrial training has been presented as a mitigation strategy which can result in more robust models. We explore the effects of two different model compression methods -- structured weight pruning and quantization -- on adversarial robustness. We show that adversarial fine-tuning of compressed models can achieve robustness performance comparable to adversarially trained models.
  arXiv  Detail & Related papers  (2024-03-14T14:34:25Z)
• Learn from the Past: A Proxy Guided Adversarial Defense Framework with Self Distillation Regularization [53.04697800214848]
  Adversarial Training (AT) is pivotal in fortifying the robustness of deep learning models. AT methods, relying on direct iterative updates for target model's defense, frequently encounter obstacles such as unstable training and catastrophic overfitting. We present a general proxy guided defense framework, LAST' (bf Learn from the Pbf ast)
  arXiv  Detail & Related papers  (2023-10-19T13:13:41Z)
• A Model Stealing Attack Against Multi-Exit Networks [13.971211573064739]
  We propose the first model stealing attack against multi-exit networks to extract both the model utility and the output strategy.<n>In experiments across multiple multi-exit networks and benchmark datasets, our method always achieves accuracy and efficiency closest to the victim models.
  arXiv  Detail & Related papers  (2023-05-23T01:24:39Z)
• Language as a Latent Sequence: deep latent variable models for semi-supervised paraphrase generation [47.33223015862104]
  We present a novel unsupervised model named variational sequence auto-encoding reconstruction (VSAR), which performs latent sequence inference given an observed text. To leverage information from text pairs, we additionally introduce a novel supervised model we call dual directional learning (DDL), which is designed to integrate with our proposed VSAR model. Our empirical evaluations suggest that the combined model yields competitive performance against the state-of-the-art supervised baselines on complete data.
  arXiv  Detail & Related papers  (2023-01-05T19:35:30Z)
• Towards Robust Recommender Systems via Triple Cooperative Defense [63.64651805384898]
  Recommender systems are often susceptible to well-crafted fake profiles, leading to biased recommendations. We propose a general framework, Triple Cooperative Defense, which cooperates to improve model robustness through the co-training of three models. Results show that the robustness improvement of TCD significantly outperforms baselines.
  arXiv  Detail & Related papers  (2022-10-25T04:45:43Z)
• A Multi-objective Memetic Algorithm for Auto Adversarial Attack Optimization Design [1.9100854225243937]
  Well-designed adversarial defense strategies can improve the robustness of deep learning models against adversarial examples. Given the defensed model, the efficient adversarial attack with less computational burden and lower robust accuracy is needed to be further exploited. We propose a multi-objective memetic algorithm for auto adversarial attack optimization design, which realizes the automatical search for the near-optimal adversarial attack towards defensed models.
  arXiv  Detail & Related papers  (2022-08-15T03:03:05Z)
• Triple Wins: Boosting Accuracy, Robustness and Efficiency Together by Enabling Input-Adaptive Inference [119.19779637025444]
  Deep networks were recently suggested to face the odds between accuracy (on clean natural images) and robustness (on adversarially perturbed images) This paper studies multi-exit networks associated with input-adaptive inference, showing their strong promise in achieving a "sweet point" in cooptimizing model accuracy, robustness and efficiency.
  arXiv  Detail & Related papers  (2020-02-24T00:40:22Z)
• Adversarial Distributional Training for Robust Deep Learning [53.300984501078126]
  Adversarial training (AT) is among the most effective techniques to improve model robustness by augmenting training data with adversarial examples. Most existing AT methods adopt a specific attack to craft adversarial examples, leading to the unreliable robustness against other unseen attacks. In this paper, we introduce adversarial distributional training (ADT), a novel framework for learning robust models.
  arXiv  Detail & Related papers  (2020-02-14T12:36:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.