GraphPFN: A Prior-Data Fitted Graph Foundation Model

• URL: http://arxiv.org/abs/2509.21489v1
• Date: Thu, 25 Sep 2025 19:47:49 GMT
• Title: GraphPFN: A Prior-Data Fitted Graph Foundation Model
• Authors: Dmitry Eremeev, Oleg Platonov, Gleb Bazhenov, Artem Babenko, Liudmila Prokhorenkova,
• Abstract summary: Foundation models pretrained on large-scale datasets have transformed such fields as natural language processing and computer vision.<n>We propose GraphPFN: a prior-data fitted network for node-level prediction.<n>On diverse real-world graph datasets with up to 50,000 nodes, GraphPFN shows strong in-context learning performance and state-of-the-art results after finetuning.
• Score: 27.47522328312435
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Foundation models pretrained on large-scale datasets have transformed such fields as natural language processing and computer vision, but their application to graph data remains limited. Recently emerged graph foundation models, such as G2T-FM, utilize tabular foundation models for graph tasks and were shown to significantly outperform prior attempts to create GFMs. However, these models primarily rely on hand-crafted graph features, limiting their ability to learn complex graph-specific patterns. In this work, we propose GraphPFN: a prior-data fitted network for node-level prediction. First, we design a prior distribution of synthetic attributed graphs. For graph structure generation, we use a novel combination of multiple stochastic block models and a preferential attachment process. We then apply graph-aware structured causal models to generate node attributes and targets. This procedure allows us to efficiently generate a wide range of realistic graph datasets. Then, we augment the tabular foundation model LimiX with attention-based graph neighborhood aggregation layers and train it on synthetic graphs sampled from our prior, allowing the model to capture graph structural dependencies not present in tabular data. On diverse real-world graph datasets with up to 50,000 nodes, GraphPFN shows strong in-context learning performance and achieves state-of-the-art results after finetuning, outperforming both G2T-FM and task-specific GNNs trained from scratch on most datasets. More broadly, our work demonstrates that pretraining on synthetic graphs from a well-designed prior distribution is an effective strategy for building graph foundation models.

Related papers

• GILT: An LLM-Free, Tuning-Free Graph Foundational Model for In-Context Learning [50.40400074353263]
  Graph Neural Networks (GNNs) are powerful tools for precessing relational data but often struggle to generalize to unseen graphs.<n>We introduce textbfGraph textbfIn-context textbfL textbfTransformer (GILT), a framework built on an LLM-free and tuning-free architecture.
  arXiv  Detail & Related papers  (2025-10-06T08:09:15Z)
• Turning Tabular Foundation Models into Graph Foundation Models [27.47522328312435]
  We propose G2T-FM, a framework for turning tabular foundation models into graph foundation models.<n>G2T-FM augments the original node features with neighborhood feature aggregation, adds structural embeddings, and then applies a TFM to the constructed node representations.<n>Our model achieves strong results, significantly outperforming publicly available GFMs and performing competitively with, and often better than, well-tuned GNNs trained from scratch.
  arXiv  Detail & Related papers  (2025-08-28T15:36:37Z)
• Deep Learning for School Dropout Detection: A Comparison of Tabular and Graph-Based Models for Predicting At-Risk Students [0.2029906424353094]
  Student dropout is a significant challenge in educational systems worldwide.<n>Graph Neural Networks (GNNs) offer a potential advantage by capturing complex relationships inherent in student data if structured as graphs.
  arXiv  Detail & Related papers  (2025-08-09T01:19:32Z)
• Scalable Graph Generative Modeling via Substructure Sequences [50.32639806800683]
  We introduce Generative Graph Pattern Machine (G$2$PM), a generative Transformer pre-training framework for graphs.<n>G$2$PM represents graph instances (nodes, edges, or entire graphs) as sequences of substructures.<n>It employs generative pre-training over the sequences to learn generalizable and transferable representations.
  arXiv  Detail & Related papers  (2025-05-22T02:16:34Z)
• Scale-Free Graph-Language Models [44.283149785253286]
  Graph-language models (GLMs) have demonstrated great potential in graph-based semi-supervised learning.<n>This paper introduces a novel GLM that integrates graph generation and text embedding within a unified framework.
  arXiv  Detail & Related papers  (2025-02-21T03:41:43Z)
• Graph Generative Pre-trained Transformer [25.611007241470645]
  This work revisits an alternative approach that represents graphs as sequences of node set and edge set.<n>We introduce the Graph Generative Pre-trained Transformer (G2PT), an auto-regressive model that learns graph structures via next-token prediction.<n>G2PT achieves superior generative performance on both generic graph and molecule datasets.
  arXiv  Detail & Related papers  (2025-01-02T05:44:11Z)
• Revisiting Graph Neural Networks on Graph-level Tasks: Comprehensive Experiments, Analysis, and Improvements [54.006506479865344]
  We propose a unified evaluation framework for graph-level Graph Neural Networks (GNNs)<n>This framework provides a standardized setting to evaluate GNNs across diverse datasets.<n>We also propose a novel GNN model with enhanced expressivity and generalization capabilities.
  arXiv  Detail & Related papers  (2025-01-01T08:48:53Z)
• An Automatic Graph Construction Framework based on Large Language Models for Recommendation [49.51799417575638]
  We introduce AutoGraph, an automatic graph construction framework based on large language models for recommendation.<n>LLMs infer the user preference and item knowledge, which is encoded as semantic vectors.<n>Latent factors are incorporated as extra nodes to link the user/item nodes, resulting in a graph with in-depth global-view semantics.
  arXiv  Detail & Related papers  (2024-12-24T07:51:29Z)
• Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees [50.78679002846741]
  We propose a novel approach to cross-task generalization in graphs via task-trees.<n>We show that pretraining a graph neural network (GNN) on diverse task-trees with a reconstruction objective induces transferable knowledge.<n>This enables efficient adaptation to downstream tasks with minimal fine-tuning.
  arXiv  Detail & Related papers  (2024-12-21T02:07:43Z)
• OpenGraph: Towards Open Graph Foundation Models [20.401374302429627]
  Graph Neural Networks (GNNs) have emerged as promising techniques for encoding structural information. Key challenge remains: the difficulty of generalizing to unseen graph data with different properties. We propose a novel graph foundation model, called OpenGraph, to address this challenge.
  arXiv  Detail & Related papers  (2024-03-02T08:05:03Z)
• GraphMaker: Can Diffusion Models Generate Large Attributed Graphs? [7.330479039715941]
  Large-scale graphs with node attributes are increasingly common in various real-world applications. Traditional graph generation methods are limited in their capacity to handle these complex structures. This paper introduces a novel diffusion model, GraphMaker, specifically designed for generating large attributed graphs.
  arXiv  Detail & Related papers  (2023-10-20T22:12:46Z)
• FIMP: Foundation Model-Informed Message Passing for Graph Neural Networks [36.648927429221466]
  Foundation-Informed Message Passing (FIMP) is a Graph Neural Network (GNN) message-passing framework. We show that the self-attention layers of foundation models can effectively be repurposed on graphs to perform cross-node attention-based message-passing.
  arXiv  Detail & Related papers  (2022-10-17T23:44:30Z)
• Graph Generative Model for Benchmarking Graph Neural Networks [73.11514658000547]
  We introduce a novel graph generative model that learns and reproduces the distribution of real-world graphs in a privacy-controlled way. Our model can successfully generate privacy-controlled, synthetic substitutes of large-scale real-world graphs that can be effectively used to benchmark GNN models.
  arXiv  Detail & Related papers  (2022-07-10T06:42:02Z)
• GraphSVX: Shapley Value Explanations for Graph Neural Networks [81.83769974301995]
  Graph Neural Networks (GNNs) achieve significant performance for various learning tasks on geometric data. In this paper, we propose a unified framework satisfied by most existing GNN explainers. We introduce GraphSVX, a post hoc local model-agnostic explanation method specifically designed for GNNs.
  arXiv  Detail & Related papers  (2021-04-18T10:40:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.