Generalize across Homophily and Heterophily: Hybrid Spectral Graph Pre-Training and Prompt Tuning

• URL: http://arxiv.org/abs/2508.11328v2
• Date: Mon, 18 Aug 2025 03:08:59 GMT
• Title: Generalize across Homophily and Heterophily: Hybrid Spectral Graph Pre-Training and Prompt Tuning
• Authors: Haitong Luo, Suhang Wang, Weiyao Zhang, Ruiqi Meng, Xuying Meng, Yujun Zhang,
• Abstract summary: Existing methods fail to handle diverse spectral distributions in real-world graphs with varying homophily.<n>We propose a novel framework that ensures spectral alignment throughout both pre-training and prompt-tuning.<n>Our experiments validate the effectiveness under both transductive and inductive learning settings.
• Score: 32.2698194084332
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph ``pre-training and prompt-tuning'' aligns downstream tasks with pre-trained objectives to enable efficient knowledge transfer under limited supervision. However, existing methods rely on homophily-based low-frequency knowledge, failing to handle diverse spectral distributions in real-world graphs with varying homophily. Our theoretical analysis reveals a spectral specificity principle: optimal knowledge transfer requires alignment between pre-trained spectral filters and the intrinsic spectrum of downstream graphs. Under limited supervision, large spectral gaps between pre-training and downstream tasks impede effective adaptation. To bridge this gap, we propose the HS-GPPT model, a novel framework that ensures spectral alignment throughout both pre-training and prompt-tuning. We utilize a hybrid spectral filter backbone and local-global contrastive learning to acquire abundant spectral knowledge. Then we design prompt graphs to align the spectral distribution with pretexts, facilitating spectral knowledge transfer across homophily and heterophily. Extensive experiments validate the effectiveness under both transductive and inductive learning settings. Our code is available at https://anonymous.4open.science/r/HS-GPPT-62D2/.

Related papers

• Advection-Diffusion on Graphs: A Bakry-Emery Laplacian for Spectral Graph Neural Networks [2.6124895681424323]
  Graph Neural Networks (GNNs) often struggle to propagate information across long distances due to oversmoothing and oversquashing.<n>We introduce a Bakry-Emery graph Laplacian that integrates diffusion and advection through a learnable node-wise potential.<n>We develop mu-ChebNet, a spectral architecture that jointly learns the potential and Chebyshev filters.
  arXiv  Detail & Related papers  (2026-02-20T11:01:12Z)
• Spectral Gating Networks [65.9496901693099]
  We introduce Spectral Gating Networks (SGN) to introduce frequency-rich expressivity in feed-forward networks.<n>SGN augments a standard activation pathway with a compact spectral pathway and learnable gates that allow the model to start from a stable base behavior.<n>It consistently improves accuracy-efficiency trade-offs under comparable computational budgets.
  arXiv  Detail & Related papers  (2026-02-07T20:00:49Z)
• MEPT: Mixture of Expert Prompt Tuning as a Manifold Mapper [75.6582687942241]
  We propose Mixture of Expert Prompt Tuning (MEPT) as an effective and efficient manifold-mapping framework.<n>MEPT integrates multiple prompt experts to adaptively learn diverse and non-stationary data distributions.<n> Empirical evaluations demonstrate that MEPT outperforms several state-of-the-art parameter efficient baselines on SuperGLUE.
  arXiv  Detail & Related papers  (2025-08-31T21:19:25Z)
• SpectrumFM: Redefining Spectrum Cognition via Foundation Modeling [65.65474629224558]
  We propose a spectrum foundation model, termed SpectrumFM, which provides a new paradigm for spectrum cognition.<n>An innovative spectrum encoder that exploits the convolutional neural networks is proposed to effectively capture both fine-grained local signal structures and high-level global dependencies in the spectrum data.<n>Two novel self-supervised learning tasks, namely masked reconstruction and next-slot signal prediction, are developed for pre-training SpectrumFM, enabling the model to learn rich and transferable representations.
  arXiv  Detail & Related papers  (2025-08-02T14:40:50Z)
• Towards Anomaly-Aware Pre-Training and Fine-Tuning for Graph Anomaly Detection [59.042018542376596]
  Graph anomaly detection (GAD) has garnered increasing attention in recent years, yet remains challenging due to two key factors.<n>Anomaly-Aware Pre-Training and Fine-Tuning (APF) is a framework to mitigate the challenges in GAD.<n> Comprehensive experiments on 10 benchmark datasets validate the superior performance of APF in comparison to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-04-19T09:57:35Z)
• Homophily Heterogeneity Matters in Graph Federated Learning: A Spectrum Sharing and Complementing Perspective [3.1700915949553843]
  We introduce the spectral Graph Neural Network (GNN) and propose a novel Federated learning method by mining Graph Spectral Properties (FedGSP)<n>On one hand, our proposed FedGSP enables clients to share generic spectral properties, allowing all clients to benefit through collaboration.<n>On the other hand, inspired by our theoretical findings, our proposed FedGSP allows clients to complement non-generic spectral properties by acquiring the spectral properties they lack.
  arXiv  Detail & Related papers  (2025-02-19T13:58:08Z)
• Holistic Physics Solver: Learning PDEs in a Unified Spectral-Physical Space [54.13671100638092]
  Holistic Physics Mixer (HPM) is a framework for integrating spectral and physical information in a unified space.<n>We show that HPM consistently outperforms state-of-the-art methods in both accuracy and computational efficiency.
  arXiv  Detail & Related papers  (2024-10-15T08:19:39Z)
• Spectral Graph Reasoning Network for Hyperspectral Image Classification [0.43512163406551996]
  Convolutional neural networks (CNNs) have achieved remarkable performance in hyperspectral image (HSI) classification. We propose a spectral graph reasoning network (SGR) learning framework comprising two crucial modules. Experiments on two HSI datasets demonstrate that the proposed architecture can significantly improve the classification accuracy.
  arXiv  Detail & Related papers  (2024-07-02T20:29:23Z)
• Shape-aware Graph Spectral Learning [36.63516222161871]
  Spectral Graph Neural Networks (GNNs) are gaining attention for their ability to surpass the limitations of message-passing GNNs. Some works empirically show that the preferred graph frequency is related to the graph homophily level. This relationship between graph frequency and graphs with homophily/heterophily has not been systematically analyzed and considered in existing spectral GNNs. We propose shape-aware regularization on a Newton Interpolation-based spectral filter that can learn an arbitrary spectral filter and incorporate prior knowledge about the desired shape of the corresponding homophily level.
  arXiv  Detail & Related papers  (2023-10-16T04:57:30Z)
• HoloNets: Spectral Convolutions do extend to Directed Graphs [59.851175771106625]
  Conventional wisdom dictates that spectral convolutional networks may only be deployed on undirected graphs. Here we show this traditional reliance on the graph Fourier transform to be superfluous. We provide a frequency-response interpretation of newly developed filters, investigate the influence of the basis used to express filters and discuss the interplay with characteristic operators on which networks are based.
  arXiv  Detail & Related papers  (2023-10-03T17:42:09Z)
• Specformer: Spectral Graph Neural Networks Meet Transformers [51.644312964537356]
  Spectral graph neural networks (GNNs) learn graph representations via spectral-domain graph convolutions. We introduce Specformer, which effectively encodes the set of all eigenvalues and performs self-attention in the spectral domain. By stacking multiple Specformer layers, one can build a powerful spectral GNN.
  arXiv  Detail & Related papers  (2023-03-02T07:36:23Z)
• Spectrum-BERT: Pre-training of Deep Bidirectional Transformers for Spectral Classification of Chinese Liquors [0.0]
  We propose a pre-training method of deep bidirectional transformers for spectral classification of Chinese liquors, abbreviated as Spectrum-BERT. We elaborately design two pre-training tasks, Next Curve Prediction (NCP) and Masked Curve Model (MCM), so that the model can effectively utilize unlabeled samples. In the comparative experiments, the proposed Spectrum-BERT significantly outperforms the baselines in multiple metrics.
  arXiv  Detail & Related papers  (2022-10-22T13:11:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.