Stability and Generalization of Adversarial Diffusion Training

• URL: http://arxiv.org/abs/2509.19234v1
• Date: Tue, 23 Sep 2025 16:55:30 GMT
• Title: Stability and Generalization of Adversarial Diffusion Training
• Authors: Hesam Hosseini, Ying Cao, Ali H. Sayed,
• Abstract summary: This work presents a stability-based generalization analysis of adversarial training under the diffusion strategy for convex losses.<n>We derive a bound showing that the generalization error grows with both the adversarial strength and the number of training steps.
• Score: 31.63120879926305
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Algorithmic stability is an established tool for analyzing generalization. While adversarial training enhances model robustness, it often suffers from robust overfitting and an enlarged generalization gap. Although recent work has established the convergence of adversarial training in decentralized networks, its generalization properties remain unexplored. This work presents a stability-based generalization analysis of adversarial training under the diffusion strategy for convex losses. We derive a bound showing that the generalization error grows with both the adversarial perturbation strength and the number of training steps, a finding consistent with single-agent case but novel for decentralized settings. Numerical experiments on logistic regression validate these theoretical predictions.

Related papers

• Generalization Analysis and Method for Domain Generalization for a Family of Recurrent Neural Networks [11.316438734182931]
  This paper proposes a method to analyze interpretability and out-of-domain (OOD) generalization for a family of recurrent neural networks (RNNs)<n>Specifically, the evolution of a trained RNN's states is modeled as an unknown, discrete-time, nonlinear closed-loop feedback system.<n>A domain generalization method is proposed that reduces the OOD generalization error and improves the robustness to distribution shifts.
  arXiv  Detail & Related papers  (2026-01-13T01:27:10Z)
• Analyzing Generalization in Pre-Trained Symbolic Regression [17.789199791229624]
  Symbolic regression algorithms search a space of mathematical expressions for formulas that explain given data.<n> Transformer-based models have emerged as a promising, promising approach shifting the expensive search to a large-scale pre-training phase.
  arXiv  Detail & Related papers  (2025-09-24T07:47:02Z)
• Generalizing to any diverse distribution: uniformity, gentle finetuning and rebalancing [55.791818510796645]
  We aim to develop models that generalize well to any diverse test distribution, even if the latter deviates significantly from the training data. Various approaches like domain adaptation, domain generalization, and robust optimization attempt to address the out-of-distribution challenge. We adopt a more conservative perspective by accounting for the worst-case error across all sufficiently diverse test distributions within a known domain.
  arXiv  Detail & Related papers  (2024-10-08T12:26:48Z)
• Out-of-distribution robustness for multivariate analysis via causal regularisation [4.487663958743944]
  We propose a regularisation strategy rooted in causality that ensures robustness against distribution shifts.<n>Building upon the anchor regression framework, we demonstrate how incorporating a straightforward regularisation term into the loss function of classical algorithms.<n>Our framework allows users to efficiently verify the compatibility of a loss function with the regularisation strategy.
  arXiv  Detail & Related papers  (2024-03-04T09:21:10Z)
• Data-Dependent Stability Analysis of Adversarial Training [18.686469222136854]
  Adrial training is the most widely used defense against adversarial example attacks. Previous training for training have not included data distribution information. Our findings demonstrate distribution shifts from data poisoning generalization.
  arXiv  Detail & Related papers  (2024-01-06T08:18:04Z)
• On robust overfitting: adversarial training induced distribution matters [32.501773057885735]
  Adversarial training may be regarded as standard training with a modified loss function. But its generalization error appears much larger than standard training under standard loss. This phenomenon, known as robust overfitting, has attracted significant research attention and remains largely as a mystery.
  arXiv  Detail & Related papers  (2023-11-28T05:11:53Z)
• Stability and Generalization of the Decentralized Stochastic Gradient Descent Ascent Algorithm [80.94861441583275]
  We investigate the complexity of the generalization bound of the decentralized gradient descent (D-SGDA) algorithm. Our results analyze the impact of different top factors on the generalization of D-SGDA. We also balance it with the generalization to obtain the optimal convex-concave setting.
  arXiv  Detail & Related papers  (2023-10-31T11:27:01Z)
• Decentralized Adversarial Training over Graphs [44.03711922549992]
  The vulnerability of machine learning models to adversarial attacks has been attracting attention in recent years.<n>We develop a decentralized adversarial framework for multiagent systems.
  arXiv  Detail & Related papers  (2023-03-23T15:05:16Z)
• Modeling Uncertain Feature Representation for Domain Generalization [49.129544670700525]
  We show that our method consistently improves the network generalization ability on multiple vision tasks. Our methods are simple yet effective and can be readily integrated into networks without additional trainable parameters or loss constraints.
  arXiv  Detail & Related papers  (2023-01-16T14:25:02Z)
• On the generalization of learning algorithms that do not converge [54.122745736433856]
  Generalization analyses of deep learning typically assume that the training converges to a fixed point. Recent results indicate that in practice, the weights of deep neural networks optimized with gradient descent often oscillate indefinitely.
  arXiv  Detail & Related papers  (2022-08-16T21:22:34Z)
• Decentralized Local Stochastic Extra-Gradient for Variational Inequalities [125.62877849447729]
  We consider distributed variational inequalities (VIs) on domains with the problem data that is heterogeneous (non-IID) and distributed across many devices. We make a very general assumption on the computational network that covers the settings of fully decentralized calculations. We theoretically analyze its convergence rate in the strongly-monotone, monotone, and non-monotone settings.
  arXiv  Detail & Related papers  (2021-06-15T17:45:51Z)
• Training Generative Adversarial Networks by Solving Ordinary Differential Equations [54.23691425062034]
  We study the continuous-time dynamics induced by GAN training. From this perspective, we hypothesise that instabilities in training GANs arise from the integration error. We experimentally verify that well-known ODE solvers (such as Runge-Kutta) can stabilise training.
  arXiv  Detail & Related papers  (2020-10-28T15:23:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.