MUG: Meta-path-aware Universal Heterogeneous Graph Pre-Training

• URL: http://arxiv.org/abs/2602.22645v1
• Date: Thu, 26 Feb 2026 05:52:28 GMT
• Title: MUG: Meta-path-aware Universal Heterogeneous Graph Pre-Training
• Authors: Lianze Shan, Jitao Zhao, Dongxiao He, Yongqi Huang, Zhiyong Feng, Weixiong Zhang,
• Abstract summary: Universal graph pre-training offers a promising way to train encoders to learn transferable representations from unlabeled graphs.<n>It remains unexplored for heterogeneous graphs, which exhibit greater structural and semantic complexity.<n>We propose a novel Meta-path-aware Universal heterogeneous Graph pre-training (MUG) approach.
• Score: 31.639971171132615
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Universal graph pre-training has emerged as a key paradigm in graph representation learning, offering a promising way to train encoders to learn transferable representations from unlabeled graphs and to effectively generalize across a wide range of downstream tasks. However, recent explorations in universal graph pre-training primarily focus on homogeneous graphs and it remains unexplored for heterogeneous graphs, which exhibit greater structural and semantic complexity. This heterogeneity makes it highly challenging to train a universal encoder for diverse heterogeneous graphs: (i) the diverse types with dataset-specific semantics hinder the construction of a unified representation space; (ii) the number and semantics of meta-paths vary across datasets, making encoding and aggregation patterns learned from one dataset difficult to apply to others. To address these challenges, we propose a novel Meta-path-aware Universal heterogeneous Graph pre-training (MUG) approach. Specifically, for challenge (i), MUG introduces a input unification module that integrates information from multiple node and relation types within each heterogeneous graph into a unified representation.This representation is then projected into a shared space by a dimension-aware encoder, enabling alignment across graphs with diverse schemas.Furthermore, for challenge (ii), MUG trains a shared encoder to capture consistent structural patterns across diverse meta-path views rather than relying on dataset-specific aggregation strategies, while a global objective encourages discriminability and reduces dataset-specific biases. Extensive experiments demonstrate the effectiveness of MUG on some real datasets.

Related papers

• Heterogeneous Graph Alignment for Joint Reasoning and Interpretability [2.2710270108565207]
  We present the Multi-Graph Meta-Transformer (MGMT), a unified, scalable, and interpretable framework for cross-graph learning.<n>MGMT first applies Graph Transformer encoders to each graph, mapping structure and attributes into a shared latent space.<n>It then selects task-relevant supernodes via attention and builds a meta-graph that connects functionally aligned supernodes across graphs using similarity in the latent space.
  arXiv  Detail & Related papers  (2026-01-30T05:40:13Z)
• OFA-MAS: One-for-All Multi-Agent System Topology Design based on Mixture-of-Experts Graph Generative Models [57.94189874119267]
  Multi-Agent Systems (MAS) offer a powerful paradigm for solving complex problems.<n>Current graph learning-based design methodologies often adhere to a "one-for-one" paradigm.<n>We propose OFA-TAD, a one-for-all framework that generates adaptive collaboration graphs for any task described in natural language.
  arXiv  Detail & Related papers  (2026-01-19T12:23:44Z)
• Transformers Provably Learn Directed Acyclic Graphs via Kernel-Guided Mutual Information [91.66597637613263]
  transformer-based models leveraging the attention mechanism have demonstrated strong empirical success in capturing complex dependencies within graphs.<n>We introduce a novel information-theoretic metric: the kernel-guided mutual information (KG-MI) based on the $f$-divergence.<n>We prove that, given sequences generated by a $K$-parent DAG, training a single-layer, multi-head transformer via a gradient ascent converges to the global optimum time.
  arXiv  Detail & Related papers  (2025-10-29T14:07:12Z)
• Multi-Relation Graph-Kernel Strengthen Network for Graph-Level Clustering [10.67474681549171]
  We propose a novel Multi-Relation Graph- Kernel Strengthen Network for Graph-Level Clustering (MGSN)<n>MGSN constructs multi-relation graphs to capture diverse semantic relationships between nodes and graphs.<n>A relation-aware representation refinement strategy is designed, which adaptively aligns multi-relation information across views.
  arXiv  Detail & Related papers  (2025-04-02T11:17:15Z)
• HiGPT: Heterogeneous Graph Language Model [27.390123898556805]
  Heterogeneous graph learning aims to capture complex relationships and diverse semantics among entities in a heterogeneous graph. Existing frameworks for heterogeneous graph learning have limitations in generalizing across diverse heterogeneous graph datasets. We propose HiGPT, a general large graph model with Heterogeneous graph instruction-tuning paradigm.
  arXiv  Detail & Related papers  (2024-02-25T08:07:22Z)
• GraphMETRO: Mitigating Complex Graph Distribution Shifts via Mixture of Aligned Experts [75.51612253852002]
  GraphMETRO is a Graph Neural Network architecture that models natural diversity and captures complex distributional shifts. GraphMETRO achieves state-of-the-art results on four datasets from the GOOD benchmark.
  arXiv  Detail & Related papers  (2023-12-07T20:56:07Z)
• GraphGLOW: Universal and Generalizable Structure Learning for Graph Neural Networks [72.01829954658889]
  This paper introduces the mathematical definition of this novel problem setting. We devise a general framework that coordinates a single graph-shared structure learner and multiple graph-specific GNNs. The well-trained structure learner can directly produce adaptive structures for unseen target graphs without any fine-tuning.
  arXiv  Detail & Related papers  (2023-06-20T03:33:22Z)
• Multi-View Graph Representation Learning Beyond Homophily [2.601278669926709]
  Unsupervised graph representation learning(GRL) aims to distill diverse graph information into task-agnostic embeddings without label supervision. A novel framework, denoted as Multi-view Graph(MVGE) is proposed, and a set of key designs are identified.
  arXiv  Detail & Related papers  (2023-04-15T08:35:49Z)
• SHGNN: Structure-Aware Heterogeneous Graph Neural Network [77.78459918119536]
  This paper proposes a novel Structure-Aware Heterogeneous Graph Neural Network (SHGNN) to address the above limitations. We first utilize a feature propagation module to capture the local structure information of intermediate nodes in the meta-path. Next, we use a tree-attention aggregator to incorporate the graph structure information into the aggregation module on the meta-path. Finally, we leverage a meta-path aggregator to fuse the information aggregated from different meta-paths.
  arXiv  Detail & Related papers  (2021-12-12T14:18:18Z)
• Multi-Level Graph Contrastive Learning [38.022118893733804]
  We propose a Multi-Level Graph Contrastive Learning (MLGCL) framework for learning robust representation of graph data by contrasting space views of graphs. The original graph is first-order approximation structure and contains uncertainty or error, while the $k$NN graph generated by encoding features preserves high-order proximity. Extensive experiments indicate MLGCL achieves promising results compared with the existing state-of-the-art graph representation learning methods on seven datasets.
  arXiv  Detail & Related papers  (2021-07-06T14:24:43Z)
• A Robust and Generalized Framework for Adversarial Graph Embedding [73.37228022428663]
  We propose a robust framework for adversarial graph embedding, named AGE. AGE generates the fake neighbor nodes as the enhanced negative samples from the implicit distribution. Based on this framework, we propose three models to handle three types of graph data.
  arXiv  Detail & Related papers  (2021-05-22T07:05:48Z)
• Learning the Implicit Semantic Representation on Graph-Structured Data [57.670106959061634]
  Existing representation learning methods in graph convolutional networks are mainly designed by describing the neighborhood of each node as a perceptual whole. We propose a Semantic Graph Convolutional Networks (SGCN) that explores the implicit semantics by learning latent semantic-paths in graphs.
  arXiv  Detail & Related papers  (2021-01-16T16:18:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.