TrustGLM: Evaluating the Robustness of GraphLLMs Against Prompt, Text, and Structure Attacks

• URL: http://arxiv.org/abs/2506.11844v1
• Date: Fri, 13 Jun 2025 14:48:01 GMT
• Title: TrustGLM: Evaluating the Robustness of GraphLLMs Against Prompt, Text, and Structure Attacks
• Authors: Qihai Zhang, Xinyue Sheng, Yuanfu Sun, Qiaoyu Tan,
• Abstract summary: We introduce TrustGLM, a comprehensive study evaluating the vulnerability of GraphLLMs to adversarial attacks across three dimensions: text, graph structure, and prompt manipulations.<n>Our findings reveal that GraphLLMs are highly susceptible to text attacks that merely replace a few semantically similar words in a node's textual attribute.<n>We also find that standard graph structure attack methods can significantly degrade model performance, while random shuffling of the candidate label set in prompt templates leads to substantial performance drops.
• Score: 3.3238054848751535
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Inspired by the success of large language models (LLMs), there is a significant research shift from traditional graph learning methods to LLM-based graph frameworks, formally known as GraphLLMs. GraphLLMs leverage the reasoning power of LLMs by integrating three key components: the textual attributes of input nodes, the structural information of node neighborhoods, and task-specific prompts that guide decision-making. Despite their promise, the robustness of GraphLLMs against adversarial perturbations remains largely unexplored-a critical concern for deploying these models in high-stakes scenarios. To bridge the gap, we introduce TrustGLM, a comprehensive study evaluating the vulnerability of GraphLLMs to adversarial attacks across three dimensions: text, graph structure, and prompt manipulations. We implement state-of-the-art attack algorithms from each perspective to rigorously assess model resilience. Through extensive experiments on six benchmark datasets from diverse domains, our findings reveal that GraphLLMs are highly susceptible to text attacks that merely replace a few semantically similar words in a node's textual attribute. We also find that standard graph structure attack methods can significantly degrade model performance, while random shuffling of the candidate label set in prompt templates leads to substantial performance drops. Beyond characterizing these vulnerabilities, we investigate defense techniques tailored to each attack vector through data-augmented training and adversarial training, which show promising potential to enhance the robustness of GraphLLMs. We hope that our open-sourced library will facilitate rapid, equitable evaluation and inspire further innovative research in this field.

Related papers

• DGP: A Dual-Granularity Prompting Framework for Fraud Detection with Graph-Enhanced LLMs [55.13817504780764]
  Real-world fraud detection applications benefit from graph learning techniques that jointly exploit node features, often rich in textual data, and graph structural information.<n>Graph-Enhanced LLMs emerge as a promising graph learning approach that converts graph information into prompts.<n>We propose Dual Granularity Prompting (DGP), which mitigates information overload by preserving fine-grained textual details for the target node.
  arXiv  Detail & Related papers  (2025-07-29T10:10:47Z)
• Navigating the Black Box: Leveraging LLMs for Effective Text-Level Graph Injection Attacks [14.181622082567124]
  This paper introduces ATAG-LLM, a novel black-box GIA framework tailored for text-attributed graphs (TAGs)<n>Our approach leverages large language models (LLMs) to generate interpretable text-level node attributes directly.<n>This method efficiently perturbs the target node with minimal training costs in a strict black-box setting, ensuring a text-level graph injection attack for TAGs.
  arXiv  Detail & Related papers  (2025-06-16T09:16:21Z)
• Learning Efficient and Generalizable Graph Retriever for Knowledge-Graph Question Answering [75.12322966980003]
  Large Language Models (LLMs) have shown strong inductive reasoning ability across various domains.<n>Most existing RAG pipelines rely on unstructured text, limiting interpretability and structured reasoning.<n>Recent studies have explored integrating knowledge graphs with LLMs for knowledge graph question answering.<n>We propose RAPL, a novel framework for efficient and effective graph retrieval in KGQA.
  arXiv  Detail & Related papers  (2025-06-11T12:03:52Z)
• GMLM: Bridging Graph Neural Networks and Language Models for Heterophilic Node Classification [0.0]
  We propose textbfGraph Masked Language Model (GMLM), a novel architecture efficiently combining Graph Neural Networks (GNNs) with Pre-trained Language Models (PLMs)<n>GMLM introduces three key innovations: (i) a textbfdynamic active node selection strategy for scalable PLM text processing; (ii) a GNN-specific textbfcontrastive pretraining stage using soft masking with a learnable graph texttt[MASK] token for robust structural representations; and (iii) a textbfdedicated fusion
  arXiv  Detail & Related papers  (2025-02-24T07:44:01Z)
• Learning on Graphs with Large Language Models(LLMs): A Deep Dive into Model Robustness [39.57155321515097]
  Large Language Models (LLMs) have demonstrated remarkable performance across various natural language processing tasks. It remains unclear whether LLMs exhibit robustness in learning on graphs.
  arXiv  Detail & Related papers  (2024-07-16T09:05:31Z)
• Graph Transductive Defense: a Two-Stage Defense for Graph Membership Inference Attacks [50.19590901147213]
  Graph neural networks (GNNs) have become instrumental in diverse real-world applications, offering powerful graph learning capabilities. GNNs are vulnerable to adversarial attacks, including membership inference attacks (MIA) This paper proposes an effective two-stage defense, Graph Transductive Defense (GTD), tailored to graph transductive learning characteristics.
  arXiv  Detail & Related papers  (2024-06-12T06:36:37Z)
• Intruding with Words: Towards Understanding Graph Injection Attacks at the Text Level [21.003091265006102]
  Graph Neural Networks (GNNs) excel across various applications but remain vulnerable to adversarial attacks. In this paper, we pioneer the exploration of Graph Injection Attacks (GIAs) at the text level. We show that text interpretability, a factor previously overlooked at the embedding level, plays a crucial role in attack strength.
  arXiv  Detail & Related papers  (2024-05-26T02:12:02Z)
• On the Adversarial Robustness of Graph Contrastive Learning Methods [9.675856264585278]
  We introduce a comprehensive evaluation robustness protocol tailored to assess the robustness of graph contrastive learning (GCL) models. We subject these models to adaptive adversarial attacks targeting the graph structure, specifically in the evasion scenario. With our work, we aim to offer insights into the robustness of GCL methods and hope to open avenues for potential future research directions.
  arXiv  Detail & Related papers  (2023-11-29T17:59:18Z)
• Towards Unsupervised Deep Graph Structure Learning [67.58720734177325]
  We propose an unsupervised graph structure learning paradigm, where the learned graph topology is optimized by data itself without any external guidance. Specifically, we generate a learning target from the original data as an "anchor graph", and use a contrastive loss to maximize the agreement between the anchor graph and the learned graph.
  arXiv  Detail & Related papers  (2022-01-17T11:57:29Z)
• Software Vulnerability Detection via Deep Learning over Disaggregated Code Graph Representation [57.92972327649165]
  This work explores a deep learning approach to automatically learn the insecure patterns from code corpora. Because code naturally admits graph structures with parsing, we develop a novel graph neural network (GNN) to exploit both the semantic context and structural regularity of a program.
  arXiv  Detail & Related papers  (2021-09-07T21:24:36Z)
• Information Obfuscation of Graph Neural Networks [96.8421624921384]
  We study the problem of protecting sensitive attributes by information obfuscation when learning with graph structured data. We propose a framework to locally filter out pre-determined sensitive attributes via adversarial training with the total variation and the Wasserstein distance.
  arXiv  Detail & Related papers  (2020-09-28T17:55:04Z)
• Graph Backdoor [53.70971502299977]
  We present GTA, the first backdoor attack on graph neural networks (GNNs) GTA departs in significant ways: it defines triggers as specific subgraphs, including both topological structures and descriptive features. It can be instantiated for both transductive (e.g., node classification) and inductive (e.g., graph classification) tasks.
  arXiv  Detail & Related papers  (2020-06-21T19:45:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.