Adversarial Graph Neural Network Benchmarks: Towards Practical and Fair Evaluation

• URL: http://arxiv.org/abs/2605.05534v1
• Date: Thu, 07 May 2026 00:27:24 GMT
• Title: Adversarial Graph Neural Network Benchmarks: Towards Practical and Fair Evaluation
• Abstract summary: Adversarial learning and the robustness of Graph Neural Networks (GNNs) are topics of widespread interest in the machine learning community.<n>We demonstrate the importance of adopting fair, robust, and standardized evaluation protocols in adversarial GNN research.
• Score: 22.639156850721648
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Adversarial learning and the robustness of Graph Neural Networks (GNNs) are topics of widespread interest in the machine learning community, as documented by the number of adversarial attacks and defenses designed for these purposes. While a rigorous evaluation of these adversarial methods is necessary to understand the robustness of GNNs in real-world applications, we posit that many works in the literature do not share the same experimental settings, leading to ambiguous and potentially contradictory scientific conclusions. In this benchmark, we demonstrate the importance of adopting fair, robust, and standardized evaluation protocols in adversarial GNN research. We perform a comprehensive re-evaluation of seven widely used attacks and eight recent defenses under both poisoning and evasion scenarios, across six popular graph datasets. Our study spans over 453,000 experiments conducted within a unified framework. We observe substantial differences in adversarial attack performance when evaluated under a fair and robust procedure. Our findings reveal that previously overlooked factors, such as target node selection and the training process of the attacked model, have a profound impact on attack effectiveness, to the extent of completely distorting performance insights. These results underscore the urgent need for standardized evaluations in adversarial graph machine learning.

Related papers

• On the Effectiveness of Adversarial Training on Malware Classifiers [14.069462668836328]
  Adversarial Training (AT) has been widely applied to harden learning-based classifiers against adversarial evasive attacks.<n>Previous work seems to suggest robustness is a task-dependent property of AT.<n>We argue it is a more complex problem that requires exploring AT and the intertwined roles played by certain factors within data.
  arXiv  Detail & Related papers  (2024-12-24T06:55:53Z)
• Graph Transductive Defense: a Two-Stage Defense for Graph Membership Inference Attacks [50.19590901147213]
  Graph neural networks (GNNs) have become instrumental in diverse real-world applications, offering powerful graph learning capabilities. GNNs are vulnerable to adversarial attacks, including membership inference attacks (MIA) This paper proposes an effective two-stage defense, Graph Transductive Defense (GTD), tailored to graph transductive learning characteristics.
  arXiv  Detail & Related papers  (2024-06-12T06:36:37Z)
• Investigating Human-Identifiable Features Hidden in Adversarial Perturbations [54.39726653562144]
  Our study explores up to five attack algorithms across three datasets. We identify human-identifiable features in adversarial perturbations. Using pixel-level annotations, we extract such features and demonstrate their ability to compromise target models.
  arXiv  Detail & Related papers  (2023-09-28T22:31:29Z)
• Comparative Evaluation of Recent Universal Adversarial Perturbations in Image Classification [27.367498200911285]
  The vulnerability of Convolutional Neural Networks (CNNs) to adversarial samples has recently garnered significant attention in the machine learning community. Recent studies have unveiled the existence of universal adversarial perturbations (UAPs) that are image-agnostic and highly transferable across different CNN models.
  arXiv  Detail & Related papers  (2023-06-20T03:29:05Z)
• On the Robustness of Random Forest Against Untargeted Data Poisoning: An Ensemble-Based Approach [42.81632484264218]
  In machine learning models, perturbations of fractions of the training set (poisoning) can seriously undermine the model accuracy. This paper aims to implement a novel hash-based ensemble approach that protects random forest against untargeted, random poisoning attacks.
  arXiv  Detail & Related papers  (2022-09-28T11:41:38Z)
• A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks [72.7373468905418]
  We develop an open-source toolkit OpenBackdoor to foster the implementations and evaluations of textual backdoor learning. We also propose CUBE, a simple yet strong clustering-based defense baseline.
  arXiv  Detail & Related papers  (2022-06-17T02:29:23Z)
• Residual Error: a New Performance Measure for Adversarial Robustness [85.0371352689919]
  A major challenge that limits the wide-spread adoption of deep learning has been their fragility to adversarial attacks. This study presents the concept of residual error, a new performance measure for assessing the adversarial robustness of a deep neural network. Experimental results using the case of image classification demonstrate the effectiveness and efficacy of the proposed residual error metric.
  arXiv  Detail & Related papers  (2021-06-18T16:34:23Z)
• Characterizing the Evasion Attackability of Multi-label Classifiers [37.00606062677375]
  Evasion attack in multi-label learning systems is an interesting, widely witnessed, yet rarely explored research topic. Characterizing the crucial factors determining the attackability of the multi-label adversarial threat is the key to interpret the origin of the vulnerability. We propose an efficient empirical attackability estimator via greedy label space exploration.
  arXiv  Detail & Related papers  (2020-12-17T07:34:40Z)
• Vulnerability Under Adversarial Machine Learning: Bias or Variance? [77.30759061082085]
  We investigate the effect of adversarial machine learning on the bias and variance of a trained deep neural network. Our analysis sheds light on why the deep neural networks have poor performance under adversarial perturbation. We introduce a new adversarial machine learning algorithm with lower computational complexity than well-known adversarial machine learning strategies.
  arXiv  Detail & Related papers  (2020-08-01T00:58:54Z)
• Benchmarking Adversarial Robustness [47.168521143464545]
  We establish a comprehensive, rigorous, and coherent benchmark to evaluate adversarial robustness on image classification tasks. Based on the evaluation results, we draw several important findings and provide insights for future research.
  arXiv  Detail & Related papers  (2019-12-26T12:37:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.