RiemannGFM: Learning a Graph Foundation Model from Riemannian Geometry

• URL: http://arxiv.org/abs/2502.03251v1
• Date: Wed, 05 Feb 2025 15:06:09 GMT
• Title: RiemannGFM: Learning a Graph Foundation Model from Riemannian Geometry
• Authors: Li Sun, Zhenhao Huang, Suyang Zhou, Qiqi Wan, Hao Peng, Philip Yu,
• Abstract summary: Graph neural networks excel at learning graph data, the omnipresent non-Euclidean structure, but often lack the generalization capacity. Recent efforts have been made to leverage Large Language Models. Key innovation is the discovery of a simple yet effective structural vocabulary of trees and cycles.
• Score: 19.299795173943476
• License:
• Abstract: The foundation model has heralded a new era in artificial intelligence, pretraining a single model to offer cross-domain transferability on different datasets. Graph neural networks excel at learning graph data, the omnipresent non-Euclidean structure, but often lack the generalization capacity. Hence, graph foundation model is drawing increasing attention, and recent efforts have been made to leverage Large Language Models. On the one hand, existing studies primarily focus on text-attributed graphs, while a wider range of real graphs do not contain fruitful textual attributes. On the other hand, the sequential graph description tailored for the Large Language Model neglects the structural complexity, which is a predominant characteristic of the graph. Such limitations motivate an important question: Can we go beyond Large Language Models, and pretrain a universal model to learn the structural knowledge for any graph? The answer in the language or vision domain is a shared vocabulary. We observe the fact that there also exist shared substructures underlying graph domain, and thereby open a new opportunity of graph foundation model with structural vocabulary. The key innovation is the discovery of a simple yet effective structural vocabulary of trees and cycles, and we explore its inherent connection to Riemannian geometry. Herein, we present a universal pretraining model, RiemannGFM. Concretely, we first construct a novel product bundle to incorporate the diverse geometries of the vocabulary. Then, on this constructed space, we stack Riemannian layers where the structural vocabulary, regardless of specific graph, is learned in Riemannian manifold offering cross-domain transferability. Extensive experiments show the effectiveness of RiemannGFM on a diversity of real graphs.

Related papers

• Towards Graph Foundation Models: The Perspective of Zero-shot Reasoning on Knowledge Graphs [14.392577069212292]
  We introduce SCORE, a unified graph reasoning framework that effectively generalizes diverse graph tasks using zero-shot learning. We evaluate SCORE using 38 diverse graph datasets, covering node-level, link-level, and graph-level tasks across multiple domains.
  arXiv  Detail & Related papers  (2024-10-16T14:26:08Z)
• Boosting Graph Foundation Model from Structural Perspective [6.387816922598151]
  We boost graph foundation model from structural perspective and propose BooG. BooG constructs virtual super nodes to unify structural characteristics of graph data from different domains. We also propose a novel pre-training objective based on contrastive learning, which learns more expressive representations for graph data.
  arXiv  Detail & Related papers  (2024-07-29T12:22:16Z)
• LSEnet: Lorentz Structural Entropy Neural Network for Deep Graph Clustering [59.89626219328127]
  Graph clustering is a fundamental problem in machine learning. Deep learning methods achieve the state-of-the-art results in recent years, but they still cannot work without predefined cluster numbers. We propose to address this problem from a fresh perspective of graph information theory.
  arXiv  Detail & Related papers  (2024-05-20T05:46:41Z)
• GraphGLOW: Universal and Generalizable Structure Learning for Graph Neural Networks [72.01829954658889]
  This paper introduces the mathematical definition of this novel problem setting. We devise a general framework that coordinates a single graph-shared structure learner and multiple graph-specific GNNs. The well-trained structure learner can directly produce adaptive structures for unseen target graphs without any fine-tuning.
  arXiv  Detail & Related papers  (2023-06-20T03:33:22Z)
• Graph Learning and Its Advancements on Large Language Models: A Holistic Survey [37.01696685233113]
  This survey focuses on the most recent advancements in integrating graph learning with pre-trained language models. We provide a holistic review that analyzes current works from the perspective of graph structure, and discusses the latest applications, trends, and challenges in graph learning.
  arXiv  Detail & Related papers  (2022-12-17T22:05:07Z)
• Hyperbolic Graph Representation Learning: A Tutorial [39.25873010585029]
  This tutorial aims to give an introduction to this emerging field of graph representation learning with the express purpose of being accessible to all audiences. We first give a brief introduction to graph representation learning as well as some preliminaryian and hyperbolic geometry. We then comprehensively revisit the technical details of the current hyperbolic graph neural networks by unifying them into a general framework.
  arXiv  Detail & Related papers  (2022-11-08T07:15:29Z)
• Explanation Graph Generation via Pre-trained Language Models: An Empirical Study with Contrastive Learning [84.35102534158621]
  We study pre-trained language models that generate explanation graphs in an end-to-end manner. We propose simple yet effective ways of graph perturbations via node and edge edit operations. Our methods lead to significant improvements in both structural and semantic accuracy of explanation graphs.
  arXiv  Detail & Related papers  (2022-04-11T00:58:27Z)
• Hyperbolic Graph Neural Networks: A Review of Methods and Applications [55.5502008501764]
  Graph neural networks generalize conventional neural networks to graph-structured data. The performance of Euclidean models in graph-related learning is still bounded and limited by the representation ability of Euclidean geometry. Recently, hyperbolic space has gained increasing popularity in processing graph data with tree-like structure and power-law distribution.
  arXiv  Detail & Related papers  (2022-02-28T15:08:48Z)
• A Self-supervised Mixed-curvature Graph Neural Network [76.3790248465522]
  We present a novel Self-supervised Mixed-curvature Graph Neural Network (SelfMGNN) We show that SelfMGNN captures the complicated graph structures in reality and outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2021-12-10T08:56:55Z)
• GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph [53.70520466556453]
  We propose GraphFormers, where layerwise GNN components are nested alongside the transformer blocks of language models. With the proposed architecture, the text encoding and the graph aggregation are fused into an iterative workflow. In addition, a progressive learning strategy is introduced, where the model is successively trained on manipulated data and original data to reinforce its capability of integrating information on graph.
  arXiv  Detail & Related papers  (2021-05-06T12:20:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.