Toward General and Robust LLM-enhanced Text-attributed Graph Learning

• URL: http://arxiv.org/abs/2504.02343v1
• Date: Thu, 03 Apr 2025 07:24:18 GMT
• Title: Toward General and Robust LLM-enhanced Text-attributed Graph Learning
• Authors: Zihao Zhang, Xunkai Li, Rong-Hua Li, Bing Zhou, Zhenjun Li, Guoren Wang,
• Abstract summary: UltraTAG is a unified pipeline for LLM-enhanced TAG learning.<n>UltraTAG-S is a robust instantiation designed to tackle the inherent sparsity issues in real-world TAGs.<n>Our experiments demonstrate that UltraTAG-S significantly outperforms existing baselines.
• Score: 29.55905028870534
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Recent advancements in Large Language Models (LLMs) and the proliferation of Text-Attributed Graphs (TAGs) across various domains have positioned LLM-enhanced TAG learning as a critical research area. By utilizing rich graph descriptions, this paradigm leverages LLMs to generate high-quality embeddings, thereby enhancing the representational capacity of Graph Neural Networks (GNNs). However, the field faces significant challenges: (1) the absence of a unified framework to systematize the diverse optimization perspectives arising from the complex interactions between LLMs and GNNs, and (2) the lack of a robust method capable of handling real-world TAGs, which often suffer from texts and edge sparsity, leading to suboptimal performance. To address these challenges, we propose UltraTAG, a unified pipeline for LLM-enhanced TAG learning. UltraTAG provides a unified comprehensive and domain-adaptive framework that not only organizes existing methodologies but also paves the way for future advancements in the field. Building on this framework, we propose UltraTAG-S, a robust instantiation of UltraTAG designed to tackle the inherent sparsity issues in real-world TAGs. UltraTAG-S employs LLM-based text propagation and text augmentation to mitigate text sparsity, while leveraging LLM-augmented node selection techniques based on PageRank and edge reconfiguration strategies to address edge sparsity. Our extensive experiments demonstrate that UltraTAG-S significantly outperforms existing baselines, achieving improvements of 2.12\% and 17.47\% in ideal and sparse settings, respectively. Moreover, as the data sparsity ratio increases, the performance improvement of UltraTAG-S also rises, which underscores the effectiveness and robustness of UltraTAG-S.

Related papers

• GDGB: A Benchmark for Generative Dynamic Text-Attributed Graph Learning [25.487795050809503]
  Generative DyTAG Benchmark (GDGB) comprises eight meticulously curated DyTAG datasets with high-quality textual features.<n>Building on GDGB, we define two novel DyTAG generation tasks: Transductive Dynamic Graph Generation (TDGG) and Inductive Dynamic Graph Generation (IDGG)
  arXiv  Detail & Related papers  (2025-07-04T02:55:32Z)
• Navigating the Black Box: Leveraging LLMs for Effective Text-Level Graph Injection Attacks [14.181622082567124]
  This paper introduces ATAG-LLM, a novel black-box GIA framework tailored for text-attributed graphs (TAGs)<n>Our approach leverages large language models (LLMs) to generate interpretable text-level node attributes directly.<n>This method efficiently perturbs the target node with minimal training costs in a strict black-box setting, ensuring a text-level graph injection attack for TAGs.
  arXiv  Detail & Related papers  (2025-06-16T09:16:21Z)
• Learning Efficient and Generalizable Graph Retriever for Knowledge-Graph Question Answering [75.12322966980003]
  Large Language Models (LLMs) have shown strong inductive reasoning ability across various domains.<n>Most existing RAG pipelines rely on unstructured text, limiting interpretability and structured reasoning.<n>Recent studies have explored integrating knowledge graphs with LLMs for knowledge graph question answering.<n>We propose RAPL, a novel framework for efficient and effective graph retrieval in KGQA.
  arXiv  Detail & Related papers  (2025-06-11T12:03:52Z)
• Model Generalization on Text Attribute Graphs: Principles with Large Language Models [14.657522068231138]
  Large language models (LLMs) have been introduced to graph learning, aiming to extend their zero-shot generalization success to tasks where labeled graph data is scarce.<n>We develop a framework for inference over text-attributed graphs (TAGs) based on task-adaptive embeddings and a generalizable graph information aggregation mechanism.<n> Evaluations on 11 real-world TAG benchmarks demonstrate that LLM-BP significantly outperforms existing approaches.
  arXiv  Detail & Related papers  (2025-02-17T14:31:00Z)
• Evaluating and Improving Graph to Text Generation with Large Language Models [46.529034150391595]
  Large language models (LLMs) have demonstrated immense potential across various tasks.<n>We conduct a comprehensive evaluation of prompting current open-source LLMs on graph-to-text generation tasks.<n>We introduce a new graph-to-text dataset, PlanGTG, annotated with two sub-tasks: reordering and attribution.
  arXiv  Detail & Related papers  (2025-01-24T13:53:54Z)
• Large Language Model-based Augmentation for Imbalanced Node Classification on Text-Attributed Graphs [13.42259312243504]
  Node classification on graphs often suffers from class imbalance, leading to biased predictions and significant risks in real-world applications.<n>We propose Large Language Model-based Augmentation on Text-Attributed Graphs (LA-TAG) to handle imbalanced node classification.
  arXiv  Detail & Related papers  (2024-10-22T10:36:15Z)
• Let's Ask GNN: Empowering Large Language Model for Graph In-Context Learning [28.660326096652437]
  We introduce AskGNN, a novel approach that bridges the gap between sequential text processing and graph-structured data. AskGNN employs a Graph Neural Network (GNN)-powered structure-enhanced retriever to select labeled nodes across graphs. Experiments across three tasks and seven LLMs demonstrate AskGNN's superior effectiveness in graph task performance.
  arXiv  Detail & Related papers  (2024-10-09T17:19:12Z)
• Parameter-Efficient Tuning Large Language Models for Graph Representation Learning [62.26278815157628]
  We introduce Graph-aware. Efficient Fine-Tuning - GPEFT, a novel approach for efficient graph representation learning. We use a graph neural network (GNN) to encode structural information from neighboring nodes into a graph prompt. We validate our approach through comprehensive experiments conducted on 8 different text-rich graphs, observing an average improvement of 2% in hit@1 and Mean Reciprocal Rank (MRR) in link prediction evaluations.
  arXiv  Detail & Related papers  (2024-04-28T18:36:59Z)
• Sequential Visual and Semantic Consistency for Semi-supervised Text Recognition [56.968108142307976]
  Scene text recognition (STR) is a challenging task that requires large-scale annotated data for training. Most existing STR methods resort to synthetic data, which may introduce domain discrepancy and degrade the performance of STR models. This paper proposes a novel semi-supervised learning method for STR that incorporates word-level consistency regularization from both visual and semantic aspects.
  arXiv  Detail & Related papers  (2024-02-24T13:00:54Z)
• LLM-DA: Data Augmentation via Large Language Models for Few-Shot Named Entity Recognition [67.96794382040547]
  $LLM-DA$ is a novel data augmentation technique based on large language models (LLMs) for the few-shot NER task. Our approach involves employing 14 contextual rewriting strategies, designing entity replacements of the same type, and incorporating noise injection to enhance robustness.
  arXiv  Detail & Related papers  (2024-02-22T14:19:56Z)
• Efficient End-to-end Language Model Fine-tuning on Graphs [21.23522552579571]
  Learning from Text-Attributed Graphs (TAGs) has attracted significant attention due to its wide range of real-world applications. We introduce LEADING, a novel and efficient approach for end-to-end fine-tuning of language models on TAGs. Our proposed approach demonstrates superior performance, achieving state-of-the-art (SOTA) results on the ogbn-arxiv leaderboard.
  arXiv  Detail & Related papers  (2023-12-07T22:35:16Z)
• Disentangled Representation Learning with Large Language Models for Text-Attributed Graphs [57.052160123387104]
  We present the Disentangled Graph-Text Learner (DGTL) model, which is able to enhance the reasoning and predicting capabilities of LLMs for TAGs. Our proposed DGTL model incorporates graph structure information through tailored disentangled graph neural network (GNN) layers. Experimental evaluations demonstrate the effectiveness of the proposed DGTL model on achieving superior or comparable performance over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-10-27T14:00:04Z)
• Leveraging Large Language Models for Node Generation in Few-Shot Learning on Text-Attributed Graphs [5.587264586806575]
  We propose a plug-and-play approach to empower text-attributed graphs through node generation using Large Language Models (LLMs) LLMs extract semantic information from labels and generate samples that belong to categories as exemplars. We employ an edge predictor to capture structural information inherent in the raw dataset and integrate the newly generated samples into the original graph.
  arXiv  Detail & Related papers  (2023-10-15T16:04:28Z)
• Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning [51.90524745663737]
  A key innovation is our use of explanations as features, which can be used to boost GNN performance on downstream tasks. Our method achieves state-of-the-art results on well-established TAG datasets. Our method significantly speeds up training, achieving a 2.88 times improvement over the closest baseline on ogbn-arxiv.
  arXiv  Detail & Related papers  (2023-05-31T03:18:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.