Reinforcement Learning Disrupts Gradient-Based Adversarial Optimization

• URL: http://arxiv.org/abs/2606.12251v1
• Date: Wed, 10 Jun 2026 15:53:36 GMT
• Title: Reinforcement Learning Disrupts Gradient-Based Adversarial Optimization
• Abstract summary: gradient-based adversarial attacks are a dominant threat to deep neural networks (DNNs)<n>We find that reinforcement learning (RL) training can disrupt the gradient structure used by attackers.<n>RL-adv achieves the highest robustness across all major attack types evaluated.
• Score: 4.530807525445671
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Gradient-based adversarial attacks remain a dominant threat to deep neural networks (DNNs), as they exploit gradient information to efficiently optimize adversarial perturbations. To address this, we investigate whether reinforcement learning (RL) training can disrupt the gradient structure used by attackers by training image classifiers with policy-gradient objectives and epsilon-greedy exploration. Through systematic experiments across CIFAR-10, CIFAR-100, and ImageNet-100 with multiple architectures, we find that RL-trained classifiers significantly disrupt gradient-based adversarial optimization. To explain this, we conduct a comprehensive mechanism analysis using loss landscape visualization, static and dynamic gradient indicators, and predictive entropy. Our analysis reveals that RL acts as an implicit regularizer, producing models with highly unstable gradient directions and smaller gradient magnitudes. This combination makes each PGD step both unreliable in direction and limited in magnitude, causing gradient-based attacks to fail within practical iteration budgets. We further show that combining RL with adversarial training (RL-adv) provides a dual-layer defense operating at two complementary levels: RL degrades gradient information available to attackers (gradient-level defense), while adversarial training strengthens decision boundaries (boundary-level defense). RL-adv achieves the highest robustness across all major attack types evaluated, including gradient-based (PGD, AutoAttack), transfer-based, and query-based attacks, outperforming SL-adv by a significant margin. These findings identify RL-induced gradient disruption as a complementary robustness mechanism and motivate future research on hybrid SL-RL training schedules that combine SL's efficiency with RL's gradient-regularization properties.

Related papers

• Information Theoretic Adversarial Training of Large Language Models [26.056920129810294]
  WARDEN is a distributionally robust adversarial training framework for large language models.<n>It dynamically reweights adversarial examples through an f -divergence ambiguity set around the empirical training distribution.<n>It substantially reduces attack success rates with computational and utility costs comparable to CAT-, CAPO-, and MixAT-based baselines.
  arXiv  Detail & Related papers  (2026-05-06T20:20:09Z)
• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• Consolidation or Adaptation? PRISM: Disentangling SFT and RL Data via Gradient Concentration [56.074760766965085]
  PRISM achieves a dynamics-aware framework that arbitrates data based on its degree of cognitive conflict with the model's existing knowledge.<n>Our findings suggest that disentangling data based on internal optimization regimes is crucial for scalable and robust agent alignment.
  arXiv  Detail & Related papers  (2026-01-12T05:43:20Z)
• Towards Reliable Evaluation of Adversarial Robustness for Spiking Neural Networks [12.939513095038977]
  Spiking Neural Networks (SNNs) utilize spike-based activations to mimic the brain's energy-efficient information processing.<n>We propose a more reliable framework for evaluating SNN adversarial robustness.
  arXiv  Detail & Related papers  (2025-12-27T08:43:06Z)
• DeepDefense: Layer-Wise Gradient-Feature Alignment for Building Robust Neural Networks [0.2099922236065961]
  DeepDefense is a novel framework that applies Gradient-Feature Alignment (GFA) regularization across multiple layers to suppress adversarial vulnerability.<n>By aligning input gradients with internal feature representations, DeepDefense promotes a smoother loss landscape in tangential directions.<n>Our approach is architecture-agnostic, simple to implement, and highly effective, offering a promising direction for improving the adversarial robustness of deep learning models.
  arXiv  Detail & Related papers  (2025-11-13T21:54:41Z)
• Adaptive Surrogate Gradients for Sequential Reinforcement Learning in Spiking Neural Networks [6.185603604308997]
  Neuromorphic computing systems are set to revolutionize energy-constrained robotics by achieving orders-of-magnitude efficiency gains.<n> Spiking Neural Networks (SNNs) represent a promising algorithmic approach for these systems, yet their application to complex control tasks faces two critical challenges.<n>We propose a novel training approach that leverages a privileged guiding policy to bootstrap the learning process, while still exploiting online environment interactions with the spiking policy.
  arXiv  Detail & Related papers  (2025-10-28T14:28:40Z)
• Robust Deep Reinforcement Learning in Robotics via Adaptive Gradient-Masked Adversarial Attacks [15.825229211045647]
  We propose the Adaptive Gradient-Masked Reinforcement (AGMR) Attack, a white-box attack method that combines DRL with a gradient-based soft masking mechanism to dynamically identify critical state dimensions and optimize adversarial policies.<n>AGMR outperforms state-of-the-art adversarial attack methods in degrading the performance of the victim agent and enhances the victim agent's robustness through adversarial defense mechanisms.
  arXiv  Detail & Related papers  (2025-03-26T15:08:58Z)
• Adversarial Training for Defense Against Label Poisoning Attacks [53.893792844055106]
  Label poisoning attacks pose significant risks to machine learning models.<n>We propose a novel adversarial training defense strategy based on support vector machines (SVMs) to counter these threats.<n>Our approach accommodates various model architectures and employs a projected gradient descent algorithm with kernel SVMs for adversarial training.
  arXiv  Detail & Related papers  (2025-02-24T13:03:19Z)
• Implicit Stochastic Gradient Descent for Training Physics-informed Neural Networks [51.92362217307946]
  Physics-informed neural networks (PINNs) have effectively been demonstrated in solving forward and inverse differential equation problems. PINNs are trapped in training failures when the target functions to be approximated exhibit high-frequency or multi-scale features. In this paper, we propose to employ implicit gradient descent (ISGD) method to train PINNs for improving the stability of training process.
  arXiv  Detail & Related papers  (2023-03-03T08:17:47Z)
• Downlink Power Allocation in Massive MIMO via Deep Learning: Adversarial Attacks and Training [62.77129284830945]
  This paper considers a regression problem in a wireless setting and shows that adversarial attacks can break the DL-based approach. We also analyze the effectiveness of adversarial training as a defensive technique in adversarial settings and show that the robustness of DL-based wireless system against attacks improves significantly.
  arXiv  Detail & Related papers  (2022-06-14T04:55:11Z)
• Learn2Perturb: an End-to-end Feature Perturbation Learning to Improve Adversarial Robustness [79.47619798416194]
  Learn2Perturb is an end-to-end feature perturbation learning approach for improving the adversarial robustness of deep neural networks. Inspired by the Expectation-Maximization, an alternating back-propagation training algorithm is introduced to train the network and noise parameters consecutively.
  arXiv  Detail & Related papers  (2020-03-02T18:27:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.