Sheaf Neural Networks on SPD Manifolds: Second-Order Geometric Representation Learning

• URL: http://arxiv.org/abs/2604.20308v1
• Date: Wed, 22 Apr 2026 08:09:04 GMT
• Title: Sheaf Neural Networks on SPD Manifolds: Second-Order Geometric Representation Learning
• Abstract summary: We develop the first sheaf neural network operating on the SPD manifold.<n>We show that SPD-valued sheaves are strictly more expressive than Euclidean sheaves.<n>Our dual-stream architecture achieves SOTA on 6/7 MoleculeNet benchmarks, with the sheaf framework providing consistent depth robustness.
• Score: 9.973930984631547
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph neural networks face two fundamental challenges rooted in the linear structure of Euclidean vector spaces: (1) Current architectures represent geometry through vectors (directions, gradients), yet many tasks require matrix-valued representations that capture relationships between directions-such as how atomic orientations covary in a molecule. These second-order representations are naturally captured by points on the symmetric positive definite matrices (SPD) manifold; (2) Standard message passing applies shared transformations across edges. Sheaf neural networks address this via edge-specific transformations, but existing formulations remain confined to vector spaces and therefore cannot propagate matrix-valued features. We address both challenges by developing the first sheaf neural network operates natively on the SPD manifold. Our key insight is that the SPD manifold admits a Lie group structure, enabling well-posed analogs of sheaf operators without projecting to Euclidean space. Theoretically, we prove that SPD-valued sheaves are strictly more expressive than Euclidean sheaves: they admit consistent configurations (global sections) that vector-valued sheaves cannot represent, directly translating to richer learned representations. Empirically, our sheaf convolution transforms effectively rank-1 directional inputs into full-rank matrices encoding local geometric structure. Our dual-stream architecture achieves SOTA on 6/7 MoleculeNet benchmarks, with the sheaf framework providing consistent depth robustness.

Related papers

• Dual-Manifold Geometry Guided Representation Learning: Adaptive Coupling between Kernel and Data Spaces [20.461783253407955]
  We introduce a dual-manifold perspective in which each convolutional layer contains two coupled geometric spaces.<n>Because these manifold share the same channel space, parameter geometry can provide complementary structural information to guide feature evolution.<n>We propose Kernel-Guided Feature Transform (KGFT), a lightweight module that derives a geometric guidance matrix from the kernel Gram matrix.
  arXiv  Detail & Related papers  (2026-08-13T02:35:08Z)
• Gaussian Sheaf Neural Networks [10.93842826346962]
  We propose a principled framework that incorporates inductive biases into graph-based learning.<n>Building on the theory of cellular sheaves, we derive a new Laplacian operator that generalizes the sheaf Laplacian to this setting and preserves its key properties.
  arXiv  Detail & Related papers  (2026-05-20T17:26:32Z)
• Deep Delta Learning [91.75868893250662]
  We introduce Deep Delta Learning (DDL), a novel architecture that generalizes the standard residual connection.<n>We provide a spectral analysis of this operator, demonstrating that the gate $(mathbfX)$ enables dynamic between identity mapping, projection, and geometric reflection.<n>This unification empowers the network to explicitly control the spectrum of its layer-wise transition operator, enabling the modeling of complex, non-monotonic dynamics.
  arXiv  Detail & Related papers  (2026-01-01T18:11:38Z)
• Geometry-Aware Deep Congruence Networks for Manifold Learning in Cross-Subject Motor Imagery [0.0]
  Cross-subject motor-imagery decoding remains a major challenge in EEG-based brain-computer interfaces.<n>We introduce novel geometry-aware preprocessing modules and deep congruence networks.<n>Our framework improves cross-subject accuracy by 3-4% over the strongest classical baselines.
  arXiv  Detail & Related papers  (2025-11-24T09:46:55Z)
• Generalized Linear Mode Connectivity for Transformers [87.32299363530996]
  A striking phenomenon is linear mode connectivity (LMC), where independently trained models can be connected by low- or zero-loss paths.<n>Prior work has predominantly focused on neuron re-ordering through permutations, but such approaches are limited in scope.<n>We introduce a unified framework that captures four symmetry classes: permutations, semi-permutations, transformations, and general invertible maps.<n>This generalization enables, for the first time, the discovery of low- and zero-barrier linear paths between independently trained Vision Transformers and GPT-2 models.
  arXiv  Detail & Related papers  (2025-06-28T01:46:36Z)
• On the Stability of Expressive Positional Encodings for Graphs [46.967035678550594]
  Using Laplacian eigenvectors as positional encodings faces two fundamental challenges. We introduce Stable and Expressive Positional generalizations (SPE) SPE is the first architecture that is (1) provably stable, and (2) universally expressive for basis invariant functions.
  arXiv  Detail & Related papers  (2023-10-04T04:48:55Z)
• Householder Projector for Unsupervised Latent Semantics Discovery [58.92485745195358]
  Householder Projector helps StyleGANs to discover more disentangled and precise semantic attributes without sacrificing image fidelity. We integrate our projector into pre-trained StyleGAN2/StyleGAN3 and evaluate the models on several benchmarks.
  arXiv  Detail & Related papers  (2023-07-16T11:43:04Z)
• Collaborative Representation for SPD Matrices with Application to Image-Set Classification [12.447073442122468]
  Collaborative representation-based classification (CRC) has demonstrated remarkable progress in the past few years. The existing CRC methods are incapable of processing the nonlinear variational information directly. Recent advances illustrate that how to effectively model these nonlinear variational information and learn invariant representations is an open challenge.
  arXiv  Detail & Related papers  (2022-01-22T04:56:53Z)
• Implicit Bias of Linear Equivariant Networks [2.580765958706854]
  Group equivariant convolutional neural networks (G-CNNs) are generalizations of convolutional neural networks (CNNs) We show that $L$-layer full-width linear G-CNNs trained via gradient descent converge to solutions with low-rank Fourier matrix coefficients.
  arXiv  Detail & Related papers  (2021-10-12T15:34:25Z)
• SLGCN: Structure Learning Graph Convolutional Networks for Graphs under Heterophily [5.619890178124606]
  We propose a structure learning graph convolutional networks (SLGCNs) to alleviate the issue from two aspects. Specifically, we design a efficient-spectral-clustering with anchors (ESC-ANCH) approach to efficiently aggregate feature representations from all similar nodes. Experimental results on a wide range of benchmark datasets illustrate that the proposed SLGCNs outperform the stat-of-the-art GNN counterparts.
  arXiv  Detail & Related papers  (2021-05-28T13:00:38Z)
• Dual-constrained Deep Semi-Supervised Coupled Factorization Network with Enriched Prior [80.5637175255349]
  We propose a new enriched prior based Dual-constrained Deep Semi-Supervised Coupled Factorization Network, called DS2CF-Net. To ex-tract hidden deep features, DS2CF-Net is modeled as a deep-structure and geometrical structure-constrained neural network. Our network can obtain state-of-the-art performance for representation learning and clustering.
  arXiv  Detail & Related papers  (2020-09-08T13:10:21Z)
• Building powerful and equivariant graph neural networks with structural message-passing [74.93169425144755]
  We propose a powerful and equivariant message-passing framework based on two ideas. First, we propagate a one-hot encoding of the nodes, in addition to the features, in order to learn a local context matrix around each node. Second, we propose methods for the parametrization of the message and update functions that ensure permutation equivariance.
  arXiv  Detail & Related papers  (2020-06-26T17:15:16Z)
• Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs [81.12344211998635]
  A common approach to define convolutions on meshes is to interpret them as a graph and apply graph convolutional networks (GCNs) We propose Gauge Equivariant Mesh CNNs which generalize GCNs to apply anisotropic gauge equivariant kernels. Our experiments validate the significantly improved expressivity of the proposed model over conventional GCNs and other methods.
  arXiv  Detail & Related papers  (2020-03-11T17:21:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.