A Unified Graph Language Model for Multi-Domain Multi-Task Graph Alignment Instruction Tuning

• URL: http://arxiv.org/abs/2605.12197v1
• Date: Tue, 12 May 2026 14:37:34 GMT
• Title: A Unified Graph Language Model for Multi-Domain Multi-Task Graph Alignment Instruction Tuning
• Abstract summary: Graph Language Models (GLMs) combine the generalization ability of Large Language Models (LLMs) with the structural modeling capacity of Graph Neural Networks (GNNs)<n>Existing GLMs that adopt GNNs as graph encoders largely overlook the problem of aligning GNN-encoded representations across domains and tasks with the LLM token space.<n>We propose UniGraphLM, a Unified Graph Language Model that incorporates a multi-domain, multi-task GNN encoder to learn generalizable graph representations aligned with textual semantics.
• Score: 29.793113892968577
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Leveraging Graph Neural Networks (GNNs) as graph encoders and aligning the resulting representations with Large Language Models (LLMs) through alignment instruction tuning has become a mainstream paradigm for constructing Graph Language Models (GLMs), combining the generalization ability of LLMs with the structural modeling capacity of GNNs. However, existing GLMs that adopt GNNs as graph encoders largely overlook the problem of aligning GNN-encoded representations across domains and tasks with the LLM token space to obtain unified graph tokens, thereby limiting their ability to generalize across diverse graph data. To bridge this gap, we aim to incorporate a multi-domain, multi-task GNN encoder into GLMs and align its representations with LLMs to enable multi-domain, multi-task graph alignment instruction tuning. This alignment problem remains underexplored and poses two key challenges: 1) learning GNN-encoded representations that are simultaneously generalizable across domains and tasks and well aligned with textual semantics is difficult, due to substantial variations in graph structures, feature distributions, and supervision signals, together with the lack of textual-semantic alignment guidance in task-specific GNN training; 2) diverse graph data and task-specific instructions can exhibit different degrees of compatibility with the LLM token space during instruction tuning, leading to varying alignment difficulty and rendering a fixed alignment strategy suboptimal. To tackle these challenges, we propose UniGraphLM, a Unified Graph Language Model that incorporates a multi-domain, multi-task GNN encoder to learn generalizable graph representations aligned with textual semantics, and then adaptively aligns these representations with the LLM.

Related papers

• GILT: An LLM-Free, Tuning-Free Graph Foundational Model for In-Context Learning [50.40400074353263]
  Graph Neural Networks (GNNs) are powerful tools for precessing relational data but often struggle to generalize to unseen graphs.<n>We introduce textbfGraph textbfIn-context textbfL textbfTransformer (GILT), a framework built on an LLM-free and tuning-free architecture.
  arXiv  Detail & Related papers  (2025-10-06T08:09:15Z)
• Masked Language Models are Good Heterogeneous Graph Generalizers [54.08788279971086]
  Masked Language Modeling-based method called LLM4HG.<n>Uses metapath-based sequences instead of HG tokens to extract structural and semantic information.
  arXiv  Detail & Related papers  (2025-06-06T15:21:24Z)
• Scalability Matters: Overcoming Challenges in InstructGLM with Similarity-Degree-Based Sampling [1.2805157669888096]
  We propose SDM-InstructGLM, a novel instruction-tuned Graph Language Model (InstructGLM) framework that enhances scalability and efficiency without relying on GNNs.<n>Our method introduces a similarity-degree-based biased random walk mechanism, which selectively samples and encodes graph information based on node-feature similarity and degree centrality.<n>Our results demonstrate the feasibility of LLM-only graph processing, enabling scalable and interpretable Graph Language Models (GLMs) optimized through instruction-based fine-tuning.
  arXiv  Detail & Related papers  (2025-05-02T06:08:21Z)
• LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Models [87.68057302738457]
  Text-Attributed Graphs (TAGs) are ubiquitous in real-world scenarios.<n>Despite large efforts to integrate Large Language Models (LLMs) and Graph Neural Networks (GNNs) for TAGs, existing approaches suffer from decoupled architectures.<n>We propose PromptGFM, a versatile GFM for TAGs grounded in graph vocabulary learning.
  arXiv  Detail & Related papers  (2025-03-05T09:45:22Z)
• GL-Fusion: Rethinking the Combination of Graph Neural Network and Large Language model [63.774726052837266]
  We introduce a new architecture that deeply integrates Graph Neural Networks (GNNs) with Large Language Models (LLMs)<n>We introduce three key innovations: (1) Structure-Aware Transformers, which incorporate GNN's message-passing capabilities directly into LLM's transformer layers; (2) Graph-Text Cross-Attention, which processes full, uncompressed text from graph nodes and edges; and (3) GNN-LLM Twin Predictor, enabling LLM's flexible autoregressive generation alongside GNN's scalable one-pass prediction.
  arXiv  Detail & Related papers  (2024-12-08T05:49:58Z)
• LLaSA: Large Language and Structured Data Assistant [27.452536284165273]
  Graph Neutral Networks (GNNs) have been introduced as an additional modality into the input of Large Language Models (LLMs)<n>We propose textbfLarge textbfLanguage and textbfStructured Data textbfAssistant (LLaSA) to enhance LLMs' ability to handle structured data.
  arXiv  Detail & Related papers  (2024-11-16T12:27:14Z)
• From Anchors to Answers: A Novel Node Tokenizer for Integrating Graph Structure into Large Language Models [27.353083085394008]
  We present NT-LLM, a novel framework with an anchor-based positional encoding scheme for graph representation.<n>Our approach strategically selects reference nodes as anchors and encodes each node's position relative to these anchors, capturing essential topological information without the computational burden of existing methods.<n>By implementing a rank-preserving objective for positional encoding pretraining, NT-LLM achieves superior performance across diverse graph tasks ranging from basic structural analysis to complex reasoning scenarios.
  arXiv  Detail & Related papers  (2024-10-14T17:21:57Z)
• All Against Some: Efficient Integration of Large Language Models for Message Passing in Graph Neural Networks [51.19110891434727]
  Large Language Models (LLMs) with pretrained knowledge and powerful semantic comprehension abilities have recently shown a remarkable ability to benefit applications using vision and text data. E-LLaGNN is a framework with an on-demand LLM service that enriches message passing procedure of graph learning by enhancing a limited fraction of nodes from the graph.
  arXiv  Detail & Related papers  (2024-07-20T22:09:42Z)
• Multi-View Empowered Structural Graph Wordification for Language Models [12.22063024099311]
  We introduce an end-to-end modality-aligning framework for LLM-graph alignment: Dual-Residual Vector Quantized-Variational AutoEncoder, namely Dr.E.<n>Our approach is purposefully designed to facilitate token-level alignment with LLMs, enabling an effective translation of the intrinsic'of graphs into comprehensible natural language.<n>Our framework ensures certain visual interpretability, efficiency, and robustness, marking the promising successful endeavor to achieve token-level alignment between LLMs and GNNs.
  arXiv  Detail & Related papers  (2024-06-19T16:43:56Z)
• LLaGA: Large Language and Graph Assistant [73.71990472543027]
  Large Language and Graph Assistant (LLaGA) is an innovative model to handle the complexities of graph-structured data. LLaGA excels in versatility, generalizability and interpretability, allowing it to perform consistently well across different datasets and tasks. Our experiments show that LLaGA delivers outstanding performance across four datasets and three tasks using one single model.
  arXiv  Detail & Related papers  (2024-02-13T02:03:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.