Leveraging Manifold Embeddings for Enhanced Graph Transformer Representations and Learning

• URL: http://arxiv.org/abs/2507.07335v1
• Date: Wed, 09 Jul 2025 23:33:36 GMT
• Title: Leveraging Manifold Embeddings for Enhanced Graph Transformer Representations and Learning
• Abstract summary: Graph transformers embed every node in a single Euclidean space.<n>We prepend a mixture-of-experts layer that routes each node to various kinds of manifold.<n>These projections provide intrinsic explanations to the latent space.
• Score: 6.436397118145477
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Graph transformers typically embed every node in a single Euclidean space, blurring heterogeneous topologies. We prepend a lightweight Riemannian mixture-of-experts layer that routes each node to various kinds of manifold, mixture of spherical, flat, hyperbolic - best matching its local structure. These projections provide intrinsic geometric explanations to the latent space. Inserted into a state-of-the-art ensemble graph transformer, this projector lifts accuracy by up to 3% on four node-classification benchmarks. The ensemble makes sure that both euclidean and non-euclidean features are captured. Explicit, geometry-aware projection thus sharpens predictive power while making graph representations more interpretable.

Related papers

• Tropical Algebraic Geometry for Neuronal Representations: An Arakelov-Green Measure Based Descriptor for Graph Learning [14.563432689352084]
  We propose a training-free geometric prior based on tropical algebraic geometry.<n>We convert spatial trees into cyclic metric graphs suitable for embedding into the Tropical Jacobian.<n>We show that the discrete Arakelov-Green measure, computed in closed form via the graph Laplacian's generalized inverse, decomposes exactly into the intrinsic path metric minus the unquantized polarization distance on this cover.
  arXiv  Detail & Related papers  (2026-08-05T05:34:50Z)
• Geometric Mixture-of-Experts with Curvature-Guided Adaptive Routing for Graph Representation Learning [11.124702719919831]
  We propose a Geometric Mixture-of-Experts framework (GeoMoE) to better accommodate multi-scale topological structures.<n>Specifically, we design a graph-aware gating network that assigns node-specific fusion weights to ensure interpretable and geometry-consistent routing.<n>Experiments on six benchmark datasets demonstrate that GeoMoE outperforms state-of-the-art baselines across diverse graph types.
  arXiv  Detail & Related papers  (2026-03-20T06:09:01Z)
• Adaptive Riemannian Graph Neural Networks [29.859977834688625]
  We introduce a novel framework that learns a continuous and anisotropic metric tensor field over the graph.<n>It allows each node to determine its optimal local geometry, enabling the model to fluidly adapt to the graph's structural landscape.<n>Our method demonstrates superior performance on both homophilic and heterophilic benchmark geometries.
  arXiv  Detail & Related papers  (2025-08-04T16:55:02Z)
• On the Expressive Power of Sparse Geometric MPNNs [3.396731589928944]
  We study the expressive power of message-passing neural networks for geometric graphs.<n>We show that generic pairs of non-isomorphic geometric graphs can be separated by message-passing networks.<n>We introduce a simple architecture, EGENNET, which achieves our theoretical guarantees and compares favorably with alternative architecture.
  arXiv  Detail & Related papers  (2024-07-02T07:48:22Z)
• Improving embedding of graphs with missing data by soft manifolds [51.425411400683565]
  The reliability of graph embeddings depends on how much the geometry of the continuous space matches the graph structure. We introduce a new class of manifold, named soft manifold, that can solve this situation. Using soft manifold for graph embedding, we can provide continuous spaces to pursue any task in data analysis over complex datasets.
  arXiv  Detail & Related papers  (2023-11-29T12:48:33Z)
• Understanding and Mitigating Hyperbolic Dimensional Collapse in Graph Contrastive Learning [70.0681902472251]
  We propose a novel contrastive learning framework to learn high-quality graph embeddings in hyperbolic space.<n>Specifically, we design the alignment metric that effectively captures the hierarchical data-invariant information.<n>We show that in the hyperbolic space one has to address the leaf- and height-level uniformity related to properties of trees.
  arXiv  Detail & Related papers  (2023-10-27T15:31:42Z)
• Curve Your Attention: Mixed-Curvature Transformers for Graph Representation Learning [77.1421343649344]
  We propose a generalization of Transformers towards operating entirely on the product of constant curvature spaces. We also provide a kernelized approach to non-Euclidean attention, which enables our model to run in time and memory cost linear to the number of nodes and edges.
  arXiv  Detail & Related papers  (2023-09-08T02:44:37Z)
• FMGNN: Fused Manifold Graph Neural Network [102.61136611255593]
  Graph representation learning has been widely studied and demonstrated effectiveness in various graph tasks. We propose the Fused Manifold Graph Neural Network (NN), a novel GNN architecture that embeds graphs into different Manifolds during training. Our experiments demonstrate that NN yields superior performance over strong baselines on the benchmarks of node classification and link prediction tasks.
  arXiv  Detail & Related papers  (2023-04-03T15:38:53Z)
• Node-Specific Space Selection via Localized Geometric Hyperbolicity in Graph Neural Networks [38.7842803074593]
  Many graph neural networks have been developed to learn graph representations in either Euclidean or hyperbolic space. In this paper, we analyze two notions of local hyperbolicity, describing the underlying local geometry. We show that our model Joint Space Graph Neural Network (JSGNN) can leverage both Euclidean and hyperbolic spaces during learning.
  arXiv  Detail & Related papers  (2023-03-03T06:04:42Z)
• Geometry Contrastive Learning on Heterogeneous Graphs [50.58523799455101]
  This paper proposes a novel self-supervised learning method, termed as Geometry Contrastive Learning (GCL) GCL views a heterogeneous graph from Euclidean and hyperbolic perspective simultaneously, aiming to make a strong merger of the ability of modeling rich semantics and complex structures. Extensive experiments on four benchmarks data sets show that the proposed approach outperforms the strong baselines.
  arXiv  Detail & Related papers  (2022-06-25T03:54:53Z)
• Neural Convolutional Surfaces [59.172308741945336]
  This work is concerned with a representation of shapes that disentangles fine, local and possibly repeating geometry, from global, coarse structures. We show that this approach achieves better neural shape compression than the state of the art, as well as enabling manipulation and transfer of shape details.
  arXiv  Detail & Related papers  (2022-04-05T15:40:11Z)
• Heterogeneous manifolds for curvature-aware graph embedding [6.3351090376024155]
  Graph embeddings are used in a broad range of Graph ML applications. The quality of such embeddings crucially depends on whether the geometry of the space matches that of the graph.
  arXiv  Detail & Related papers  (2022-02-02T18:18:35Z)
• 5* Knowledge Graph Embeddings with Projective Transformations [13.723120574076127]
  We present a novel knowledge graph embedding model (5*E) in projective geometry. It supports multiple simultaneous transformations - specifically inversion, reflection, translation, rotation, and homothety. It outperforms existing approaches on the most widely used link prediction benchmarks.
  arXiv  Detail & Related papers  (2020-06-08T23:28:07Z)
• Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs [81.12344211998635]
  A common approach to define convolutions on meshes is to interpret them as a graph and apply graph convolutional networks (GCNs) We propose Gauge Equivariant Mesh CNNs which generalize GCNs to apply anisotropic gauge equivariant kernels. Our experiments validate the significantly improved expressivity of the proposed model over conventional GCNs and other methods.
  arXiv  Detail & Related papers  (2020-03-11T17:21:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.