LOHA: Direct Graph Spectral Contrastive Learning Between Low-pass and High-pass Views

• URL: http://arxiv.org/abs/2501.02969v1
• Date: Mon, 06 Jan 2025 12:25:02 GMT
• Title: LOHA: Direct Graph Spectral Contrastive Learning Between Low-pass and High-pass Views
• Authors: Ziyun Zou, Yinghui Jiang, Lian Shen, Juan Liu, Xiangrong Liu,
• Abstract summary: Spectral Graph Neural Networks effectively handle graphs with different homophily levels, with low-pass filter mining feature smoothness and high-pass filter capturing differences.<n>When distinct filters could naturally form two opposite views for self-supervised learning, the commonalities between the counterparts for the same node remain unexplored.<n>This paper proposes a simple yet effective self-supervised contrastive framework, LOHA, to address this gap.
• Score: 3.0930990296129828
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spectral Graph Neural Networks effectively handle graphs with different homophily levels, with low-pass filter mining feature smoothness and high-pass filter capturing differences. When these distinct filters could naturally form two opposite views for self-supervised learning, the commonalities between the counterparts for the same node remain unexplored, leading to suboptimal performance. In this paper, a simple yet effective self-supervised contrastive framework, LOHA, is proposed to address this gap. LOHA optimally leverages low-pass and high-pass views by embracing "harmony in diversity". Rather than solely maximizing the difference between these distinct views, which may lead to feature separation, LOHA harmonizes the diversity by treating the propagation of graph signals from both views as a composite feature. Specifically, a novel high-dimensional feature named spectral signal trend is proposed to serve as the basis for the composite feature, which remains relatively unaffected by changing filters and focuses solely on original feature differences. LOHA achieves an average performance improvement of 2.8% over runner-up models on 9 real-world datasets with varying homophily levels. Notably, LOHA even surpasses fully-supervised models on several datasets, which underscores the potential of LOHA in advancing the efficacy of spectral GNNs for diverse graph structures.

Related papers

• Enhancing Spectral Graph Neural Networks with LLM-Predicted Homophily [48.135717446964385]
  Spectral Graph Neural Networks (SGNNs) have achieved remarkable performance in tasks such as node classification.<n>We propose a novel framework that leverages Large Language Models (LLMs) to estimate the homophily level of a graph.<n>Our framework consistently improves performance over strong SGNN baselines.
  arXiv  Detail & Related papers  (2025-06-17T06:17:19Z)
• Efficient Identity and Position Graph Embedding via Spectral-Based Random Feature Aggregation [37.25217644507099]
  Graph neural networks (GNNs) capture graph structures via a feature aggregation mechanism.<n>It is unclear for most GNN-based methods which property they can capture.<n>We propose random feature aggregation (RFA) for efficient identity and position embedding.
  arXiv  Detail & Related papers  (2025-05-27T10:26:15Z)
• A Pre-Training and Adaptive Fine-Tuning Framework for Graph Anomaly Detection [67.77204352386897]
  Graph anomaly detection (GAD) has garnered increasing attention in recent years, yet it remains challenging due to the scarcity of abnormal nodes and the high cost of label annotations. We propose PAF, a framework specifically designed for GAD that combines low- and high-pass filters in the pre-training phase to capture the full spectrum of frequency information in node features.
  arXiv  Detail & Related papers  (2025-04-19T09:57:35Z)
• Collaborative Filtering Meets Spectrum Shift: Connecting User-Item Interaction with Graph-Structured Side Information [11.650613484855356]
  Graph Neural Network (GNN) has demonstrated their superiority in collaborative filtering. However, when graph-structured side information is integrated into the U-I bipartite graph, existing graph collaborative filtering methods fall short of achieving satisfactory performance. We propose Spectrum Shift Correction (dubbed SSC), incorporating shifting and scaling factors to enable spectral GNNs to adapt to the shifted spectrum.
  arXiv  Detail & Related papers  (2025-02-12T02:24:26Z)
• Dual-Frequency Filtering Self-aware Graph Neural Networks for Homophilic and Heterophilic Graphs [60.82508765185161]
  We propose Dual-Frequency Filtering Self-aware Graph Neural Networks (DFGNN) DFGNN integrates low-pass and high-pass filters to extract smooth and detailed topological features. It dynamically adjusts filtering ratios to accommodate both homophilic and heterophilic graphs.
  arXiv  Detail & Related papers  (2024-11-18T04:57:05Z)
• Addressing Graph Heterogeneity and Heterophily from A Spectral Perspective [46.37860909753809]
  Heterogeneity refers to a graph with multiple types of nodes or edges, while heterophily refers to the fact that connected nodes are more likely to have dissimilar attributes or labels. We propose a Heterogeneous Heterophilic Spectral Graph Neural Network (H2SGNN), which employs two modules: local independent filtering and global hybrid filtering. Extensive experiments are conducted on four datasets to validate the effectiveness of the proposed H2SGNN.
  arXiv  Detail & Related papers  (2024-10-17T09:23:53Z)
• LAMP: Learnable Meta-Path Guided Adversarial Contrastive Learning for Heterogeneous Graphs [22.322402072526927]
  Heterogeneous Graph Contrastive Learning (HGCL) usually requires pre-defined meta-paths. textsfLAMP integrates various meta-path sub-graphs into a unified and stable structure. textsfLAMP significantly outperforms existing state-of-the-art unsupervised models in terms of accuracy and robustness.
  arXiv  Detail & Related papers  (2024-09-10T08:27:39Z)
• Node-wise Filtering in Graph Neural Networks: A Mixture of Experts Approach [58.8524608686851]
  Graph Neural Networks (GNNs) have proven to be highly effective for node classification tasks across diverse graph structural patterns. Traditionally, GNNs employ a uniform global filter, typically a low-pass filter for homophilic graphs and a high-pass filter for heterophilic graphs. We introduce a novel GNN framework Node-MoE that utilizes a mixture of experts to adaptively select the appropriate filters for different nodes.
  arXiv  Detail & Related papers  (2024-06-05T17:12:38Z)
• GPatcher: A Simple and Adaptive MLP Model for Alleviating Graph Heterophily [15.93465948768545]
  We demystify the impact of graph heterophily on graph neural networks (GNNs) filters. We propose a simple yet powerful GNN named GPatcher by leveraging the patch-Mixer architectures. Our model demonstrates outstanding performance on node classification compared with popular homophily GNNs and state-of-the-art heterophily GNNs.
  arXiv  Detail & Related papers  (2023-06-25T20:57:35Z)
• Single-Pass Contrastive Learning Can Work for Both Homophilic and Heterophilic Graph [60.28340453547902]
  Graph contrastive learning (GCL) techniques typically require two forward passes for a single instance to construct the contrastive loss. Existing GCL approaches fail to provide strong performance guarantees. We implement the Single-Pass Graph Contrastive Learning method (SP-GCL) Empirically, the features learned by the SP-GCL can match or outperform existing strong baselines with significantly less computational overhead.
  arXiv  Detail & Related papers  (2022-11-20T07:18:56Z)
• Optimal Propagation for Graph Neural Networks [51.08426265813481]
  We propose a bi-level optimization approach for learning the optimal graph structure. We also explore a low-rank approximation model for further reducing the time complexity.
  arXiv  Detail & Related papers  (2022-05-06T03:37:00Z)
• Quasi-Framelets: Robust Graph Neural Networks via Adaptive Framelet Convolution [28.474359021962346]
  We propose a multiscale framelet convolution for spectral graph neural networks (GNNs) The proposed design excels in filtering out unwanted spectral information and significantly reduces the adverse effects of noisy graph signals. It exhibits remarkable resilience to noisy data and adversarial attacks, highlighting its potential as a robust solution for real-world graph applications.
  arXiv  Detail & Related papers  (2022-01-11T00:10:28Z)
• Message Passing in Graph Convolution Networks via Adaptive Filter Banks [81.12823274576274]
  We present a novel graph convolution operator, termed BankGCN. It decomposes multi-channel signals on graphs into subspaces and handles particular information in each subspace with an adapted filter. It achieves excellent performance in graph classification on a collection of benchmark graph datasets.
  arXiv  Detail & Related papers  (2021-06-18T04:23:34Z)
• Graph Neural Networks with Adaptive Frequency Response Filter [55.626174910206046]
  We develop a graph neural network framework AdaGNN with a well-smooth adaptive frequency response filter. We empirically validate the effectiveness of the proposed framework on various benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-26T19:31:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.