Size Transferability of Graph Transformers with Convolutional Positional Encodings

• URL: http://arxiv.org/abs/2602.15239v1
• Date: Mon, 16 Feb 2026 22:38:56 GMT
• Title: Size Transferability of Graph Transformers with Convolutional Positional Encodings
• Authors: Javier Porras-Valenzuela, Zhiyang Wang, Alejandro Ribeiro,
• Abstract summary: Graph Transformers (GTs) are attention-based architectures for graph-structured data.<n>We study GTs through the lens of manifold limit models for graph sequences.<n>We show that GTs inherit transferability guarantees from their positional encodings.
• Score: 82.27361992510494
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transformers have achieved remarkable success across domains, motivating the rise of Graph Transformers (GTs) as attention-based architectures for graph-structured data. A key design choice in GTs is the use of Graph Neural Network (GNN)-based positional encodings to incorporate structural information. In this work, we study GTs through the lens of manifold limit models for graph sequences and establish a theoretical connection between GTs with GNN positional encodings and Manifold Neural Networks (MNNs). Building on transferability results for GNNs under manifold convergence, we show that GTs inherit transferability guarantees from their positional encodings. In particular, GTs trained on small graphs provably generalize to larger graphs under mild assumptions. We complement our theory with extensive experiments on standard graph benchmarks, demonstrating that GTs exhibit scalable behavior on par with GNNs. To further show the efficiency in a real-world scenario, we implement GTs for shortest path distance estimation over terrains to better illustrate the efficiency of the transferable GTs. Our results provide new insights into the understanding of GTs and suggest practical directions for efficient training of GTs in large-scale settings.

Related papers

• OpenGT: A Comprehensive Benchmark For Graph Transformers [13.214504021335749]
  Graph Transformers (GTs) have recently demonstrated remarkable performance across diverse domains.<n>This paper introduces OpenGT, a comprehensive benchmark for Graph Transformers.
  arXiv  Detail & Related papers  (2025-06-05T08:48:46Z)
• Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence [20.305592785919647]
  We explore the untapped potential of Graph Neural Networks (GNNs) through an enhanced framework, GNN+.<n>We conduct a systematic re-evaluation of three classic GNNs enhanced by the GNN+ framework across 14 well-known graph-level datasets.<n>Our results reveal that, contrary to prevailing beliefs, these classic GNNs consistently match or surpass the performance of GTs.
  arXiv  Detail & Related papers  (2025-02-13T12:24:23Z)
• Benchmarking Positional Encodings for GNNs and Graph Transformers [20.706469085872516]
  We present a benchmark of Positional s (PEs) in a unified framework that includes both message-passing GNNs and GTs. We also establish theoretical connections between MPNNs and GTs and introduce a sparsified GRIT attention mechanism to examine the influence of global connectivity.
  arXiv  Detail & Related papers  (2024-11-19T18:57:01Z)
• Rethinking Graph Transformer Architecture Design for Node Classification [4.497245600377944]
  Graph Transformer (GT) is a special type of Graph Neural Networks (GNNs) that utilize multi-head attention to facilitate high-order message passing. In this work, we conduct observational experiments to explore the adaptability of the GT architecture in node classification tasks. Our proposed GT architecture can effectively adapt to node classification tasks without being affected by global noise and computational efficiency limitations.
  arXiv  Detail & Related papers  (2024-10-15T02:08:16Z)
• A Pure Transformer Pretraining Framework on Text-attributed Graphs [50.833130854272774]
  We introduce a feature-centric pretraining perspective by treating graph structure as a prior. Our framework, Graph Sequence Pretraining with Transformer (GSPT), samples node contexts through random walks. GSPT can be easily adapted to both node classification and link prediction, demonstrating promising empirical success on various datasets.
  arXiv  Detail & Related papers  (2024-06-19T22:30:08Z)
• Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification [7.14327815822376]
  Graph Transformers (GTs) have emerged as popular alternatives to traditional Graph Neural Networks (GNNs) In this paper, we reevaluate the performance of three classic GNN models (GCN, GAT, and GraphSAGE) against GTs.
  arXiv  Detail & Related papers  (2024-06-13T10:53:33Z)
• A Manifold Perspective on the Statistical Generalization of Graph Neural Networks [84.01980526069075]
  We take a manifold perspective to establish the statistical generalization theory of GNNs on graphs sampled from a manifold in the spectral domain.<n>We prove that the generalization bounds of GNNs decrease linearly with the size of the graphs in the logarithmic scale, and increase linearly with the spectral continuity constants of the filter functions.
  arXiv  Detail & Related papers  (2024-06-07T19:25:02Z)
• Graph Transformers for Large Graphs [57.19338459218758]
  This work advances representation learning on single large-scale graphs with a focus on identifying model characteristics and critical design constraints. A key innovation of this work lies in the creation of a fast neighborhood sampling technique coupled with a local attention mechanism. We report a 3x speedup and 16.8% performance gain on ogbn-products and snap-patents, while we also scale LargeGT on ogbn-100M with a 5.9% performance improvement.
  arXiv  Detail & Related papers  (2023-12-18T11:19:23Z)
• Exploring Sparsity in Graph Transformers [67.48149404841925]
  Graph Transformers (GTs) have achieved impressive results on various graph-related tasks. However, the huge computational cost of GTs hinders their deployment and application, especially in resource-constrained environments. We propose a comprehensive textbfGraph textbfTransformer textbfSParsification (GTSP) framework that helps to reduce the computational complexity of GTs.
  arXiv  Detail & Related papers  (2023-12-09T06:21:44Z)
• T-GAE: Transferable Graph Autoencoder for Network Alignment [79.89704126746204]
  T-GAE is a graph autoencoder framework that leverages transferability and stability of GNNs to achieve efficient network alignment without retraining. Our experiments demonstrate that T-GAE outperforms the state-of-the-art optimization method and the best GNN approach by up to 38.7% and 50.8%, respectively.
  arXiv  Detail & Related papers  (2023-10-05T02:58:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.