GIST: Gauge-Invariant Spectral Transformers for Scalable Graph Neural Operators

• URL: http://arxiv.org/abs/2603.16849v1
• Date: Tue, 17 Mar 2026 17:54:26 GMT
• Title: GIST: Gauge-Invariant Spectral Transformers for Scalable Graph Neural Operators
• Abstract summary: GIST (Gauge-Invariant Spectral Transformers) is a new graph transformer architecture that achieves end-to-end $mathcalO(N)$ complexity through random projections.<n>We prove GIST achieves discretization-invariant learning with bounded mismatch error, enabling parameter transfer across arbitrary mesh resolutions for neural operator applications.
• Score: 9.058225799011545
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Adapting transformer positional encoding to meshes and graph-structured data presents significant computational challenges: exact spectral methods require cubic-complexity eigendecomposition and can inadvertently break gauge invariance through numerical solver artifacts, while efficient approximate methods sacrifice gauge symmetry by design. Both failure modes cause catastrophic generalization in inductive learning, where models trained with one set of numerical choices fail when encountering different spectral decompositions of similar graphs or discretizations of the same mesh. We propose GIST (Gauge-Invariant Spectral Transformers), a new graph transformer architecture that resolves this challenge by achieving end-to-end $\mathcal{O}(N)$ complexity through random projections while algorithmically preserving gauge invariance via inner-product-based attention on the projected embeddings. We prove GIST achieves discretization-invariant learning with bounded mismatch error, enabling parameter transfer across arbitrary mesh resolutions for neural operator applications. Empirically, GIST matches state-of-the-art on standard graph benchmarks (e.g., achieving 99.50% micro-F1 on PPI) while uniquely scaling to mesh-based Neural Operator benchmarks with up to 750K nodes, achieving state-of-the-art aerodynamic prediction on the challenging DrivAerNet and DrivAerNet++ datasets.

Related papers

• Directed Graph Topology Inference via Graph Filter Identification [54.541275287889164]
  We address the problem of inferring a directed network from nodal measurements generated by a graph convolutional filter.<n>A joint graph filter and topology identification algorithm is also proposed, which alternates between the aforementioned steps to offer improved sample complexity.
  arXiv  Detail & Related papers  (2026-06-25T18:25:57Z)
• Breaking Symmetry Bottlenecks in GNN Readouts [0.764671395172401]
  Graph neural networks (GNNs) are widely used for learning on structured data, yet their ability to distinguish non-isomorphic graphs is fundamentally limited.<n>We show that an independent bottleneck arises at the readout stage.<n>We introduce projector-based invariant readouts that decompose node representations into symmetry-aware channels.
  arXiv  Detail & Related papers  (2026-02-05T18:08:13Z)
• Plain Transformers are Surprisingly Powerful Link Predictors [57.01966734467712]
  Link prediction is a core challenge in graph machine learning, demanding models that capture rich and complex topological dependencies.<n>While Graph Neural Networks (GNNs) are the standard solution, state-of-the-art pipelines often rely on explicit structurals or memory-intensive node embeddings.<n>We present PENCIL, an encoder-only plain Transformer that replaces hand-crafted priors with attention over sampled local subgraphs.
  arXiv  Detail & Related papers  (2026-02-02T02:45:52Z)
• LARGE: A Locally Adaptive Regularization Approach for Estimating Gaussian Graphical Models [2.3696387635465608]
  We develop Locally Adaptive Regularization for Graph Estimation (LARGE)<n>LARGE is an approach to adaptively learn nodewise tuning parameters to improve graph estimation and selection.<n>We demonstrate the practical utility of our method by estimating brain connectivity from a real fMRI data set.
  arXiv  Detail & Related papers  (2026-01-14T18:37:50Z)
• A Spectral Interpretation of Redundancy in a Graph Reservoir [51.40366905583043]
  This work revisits the definition of the reservoir in the Multiresolution Reservoir Graph Neural Network (MRGNN)<n>It proposes a variant based on a Fairing algorithm originally introduced in the field of surface design in computer graphics.<n>The core contribution of the paper lies in the theoretical analysis of the algorithm from a random walks perspective.
  arXiv  Detail & Related papers  (2025-07-17T10:02:57Z)
• GITO: Graph-Informed Transformer Operator for Learning Complex Partial Differential Equations [0.0]
  We present a novel graph-informed transformer operator (GITO) architecture for learning complex partial differential equation systems.<n>GITO consists of two main modules: a hybrid graph transformer (HGT) and a transformer neural operator (TNO)<n> Empirical results on benchmark PDE tasks demonstrate that GITO outperforms existing transformer-based neural operators.
  arXiv  Detail & Related papers  (2025-06-16T18:35:45Z)
• ASWT-SGNN: Adaptive Spectral Wavelet Transform-based Self-Supervised Graph Neural Network [20.924559944655392]
  This paper proposes an Adaptive Spectral Wavelet Transform-based Self-Supervised Graph Neural Network (ASWT-SGNN) ASWT-SGNN accurately approximates the filter function in high-density spectral regions, avoiding costly eigen-decomposition. It achieves comparable performance to state-of-the-art models in node classification tasks.
  arXiv  Detail & Related papers  (2023-12-10T03:07:42Z)
• Discrete Graph Auto-Encoder [52.50288418639075]
  We introduce a new framework named Discrete Graph Auto-Encoder (DGAE) We first use a permutation-equivariant auto-encoder to convert graphs into sets of discrete latent node representations. In the second step, we sort the sets of discrete latent representations and learn their distribution with a specifically designed auto-regressive model.
  arXiv  Detail & Related papers  (2023-06-13T12:40:39Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• SF-SGL: Solver-Free Spectral Graph Learning from Linear Measurements [16.313489255657203]
  spectral graph densification framework (SGL) for learning resistor networks with linear measurements. solver-free method (SF-SGL) that exploits multilevel spectral approximation of the graphs. EDA algorithm for vectorless power/thermal integrity verifications.
  arXiv  Detail & Related papers  (2023-02-09T00:33:19Z)
• Multipole Graph Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  One of the main challenges in using deep learning-based methods for simulating physical systems is formulating physics-based data. We propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.
  arXiv  Detail & Related papers  (2020-06-16T21:56:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.