If You Want to Be Robust, Be Wary of Initialization

• URL: http://arxiv.org/abs/2510.22652v1
• Date: Sun, 26 Oct 2025 12:28:12 GMT
• Title: If You Want to Be Robust, Be Wary of Initialization
• Authors: Sofiane Ennadir, Johannes F. Lutzeyer, Michalis Vazirgiannis, El Houcine Bergou,
• Abstract summary: Graph Neural Networks (GNNs) have demonstrated remarkable performance across a spectrum of graph-related tasks.<n>However, concerns persist regarding their vulnerability to adversarial perturbations.<n>We introduce a theoretical framework bridging the connection between initialization strategies and a network's resilience to adversarial perturbations.<n>Our analysis reveals a direct relationship between initial weights, number of training epochs and the model's vulnerability.
• Score: 28.195617869726636
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Graph Neural Networks (GNNs) have demonstrated remarkable performance across a spectrum of graph-related tasks, however concerns persist regarding their vulnerability to adversarial perturbations. While prevailing defense strategies focus primarily on pre-processing techniques and adaptive message-passing schemes, this study delves into an under-explored dimension: the impact of weight initialization and associated hyper-parameters, such as training epochs, on a model's robustness. We introduce a theoretical framework bridging the connection between initialization strategies and a network's resilience to adversarial perturbations. Our analysis reveals a direct relationship between initial weights, number of training epochs and the model's vulnerability, offering new insights into adversarial robustness beyond conventional defense mechanisms. While our primary focus is on GNNs, we extend our theoretical framework, providing a general upper-bound applicable to Deep Neural Networks. Extensive experiments, spanning diverse models and real-world datasets subjected to various adversarial attacks, validate our findings. We illustrate that selecting appropriate initialization not only ensures performance on clean datasets but also enhances model robustness against adversarial perturbations, with observed gaps of up to 50\% compared to alternative initialization approaches.

Related papers

• Adversarial generalization of unfolding (model-based) networks [0.0]
  We study the adversarial generalization of unfolding networks when perturbed with $l$-norm constrained attacks.<n>This is the first theoretical analysis on the adversarial generalization of unfolding networks.<n>We present a series of experiments on real-world data, with results corroborating our derived theory.
  arXiv  Detail & Related papers  (2025-09-18T19:17:07Z)
• Unifying Adversarial Perturbation for Graph Neural Networks [46.74046710620528]
  This paper studies the vulnerability of Graph Neural Networks (GNNs) to adversarial attacks on node features and graph structure.<n>We propose PerturbEmbedding, that integrates adversarial perturbation and training, enhancing GNNs' resilience to such attacks.<n>We demonstrate that PerturbEmbedding significantly improves both the robustness and generalization abilities of GNNs, outperforming existing methods.
  arXiv  Detail & Related papers  (2025-08-30T06:53:36Z)
• How Learning Dynamics Drive Adversarially Robust Generalization? [3.7919737164481284]
  We propose a novel PAC-Bayesian framework that explicitly links adversarial robustness to the posterior covariance of model parameters and the curvature of the adversarial loss landscape.<n>Our analyses reveal how key factors, such as learning rate, gradient noise, and Hessian structure, jointly shape robust generalization during training.
  arXiv  Detail & Related papers  (2024-10-10T08:34:43Z)
• GAIM: Attacking Graph Neural Networks via Adversarial Influence Maximization [12.651079126612121]
  We present an integrated adversarial attack method conducted on a node basis while considering the strict black-box setting. In our approach, we unify the selection of the target node and the construction of feature perturbations into a single optimization problem. We extend our method to accommodate label-oriented attacks, broadening its applicability.
  arXiv  Detail & Related papers  (2024-08-20T15:41:20Z)
• Problem space structural adversarial attacks for Network Intrusion Detection Systems based on Graph Neural Networks [8.629862888374243]
  We propose the first formalization of adversarial attacks specifically tailored for GNN in network intrusion detection. We outline and model the problem space constraints that attackers need to consider to carry out feasible structural attacks in real-world scenarios. Our findings demonstrate the increased robustness of the models against classical feature-based adversarial attacks.
  arXiv  Detail & Related papers  (2024-03-18T14:40:33Z)
• Doubly Robust Instance-Reweighted Adversarial Training [107.40683655362285]
  We propose a novel doubly-robust instance reweighted adversarial framework. Our importance weights are obtained by optimizing the KL-divergence regularized loss function. Our proposed approach outperforms related state-of-the-art baseline methods in terms of average robust performance.
  arXiv  Detail & Related papers  (2023-08-01T06:16:18Z)
• TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalization [89.54947228958494]
  This paper focuses on the fine-tuning of an adversarially pre-trained model in various classification tasks. We propose a novel statistics-based approach, Two-WIng NormliSation (TWINS) fine-tuning framework. TWINS is shown to be effective on a wide range of image classification datasets in terms of both generalization and robustness.
  arXiv  Detail & Related papers  (2023-03-20T14:12:55Z)
• Building Robust Ensembles via Margin Boosting [98.56381714748096]
  In adversarial robustness, a single model does not usually have enough power to defend against all possible adversarial attacks. We develop an algorithm for learning an ensemble with maximum margin. We show that our algorithm not only outperforms existing ensembling techniques, but also large models trained in an end-to-end fashion.
  arXiv  Detail & Related papers  (2022-06-07T14:55:58Z)
• Self-Ensemble Adversarial Training for Improved Robustness [14.244311026737666]
  Adversarial training is the strongest strategy against various adversarial attacks among all sorts of defense methods. Recent works mainly focus on developing new loss functions or regularizers, attempting to find the unique optimal point in the weight space. We devise a simple but powerful emphSelf-Ensemble Adversarial Training (SEAT) method for yielding a robust classifier by averaging weights of history models.
  arXiv  Detail & Related papers  (2022-03-18T01:12:18Z)
• Non-Singular Adversarial Robustness of Neural Networks [58.731070632586594]
  Adrial robustness has become an emerging challenge for neural network owing to its over-sensitivity to small input perturbations. We formalize the notion of non-singular adversarial robustness for neural networks through the lens of joint perturbations to data inputs as well as model weights.
  arXiv  Detail & Related papers  (2021-02-23T20:59:30Z)
• Developing Constrained Neural Units Over Time [81.19349325749037]
  This paper focuses on an alternative way of defining Neural Networks, that is different from the majority of existing approaches. The structure of the neural architecture is defined by means of a special class of constraints that are extended also to the interaction with data. The proposed theory is cast into the time domain, in which data are presented to the network in an ordered manner.
  arXiv  Detail & Related papers  (2020-09-01T09:07:25Z)
• Graph Backdoor [53.70971502299977]
  We present GTA, the first backdoor attack on graph neural networks (GNNs) GTA departs in significant ways: it defines triggers as specific subgraphs, including both topological structures and descriptive features. It can be instantiated for both transductive (e.g., node classification) and inductive (e.g., graph classification) tasks.
  arXiv  Detail & Related papers  (2020-06-21T19:45:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.