Spectral-Aware Augmentation for Enhanced Graph Representation Learning

• URL: http://arxiv.org/abs/2310.13845v2
• Date: Wed, 4 Sep 2024 23:17:41 GMT
• Title: Spectral-Aware Augmentation for Enhanced Graph Representation Learning
• Authors: Kaiqi Yang, Haoyu Han, Wei Jin, Hui Liu,
• Abstract summary: We present GASSER, a model that applies tailored perturbations to specific frequencies of graph structures in the spectral domain. Through extensive experimentation and theoretical analysis, we demonstrate that the augmentation views generated by GASSER are adaptive, controllable, and intuitively aligned with the homophily ratios and spectrum of graph structures.
• Score: 10.36458924914831
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Graph Contrastive Learning (GCL) has demonstrated remarkable effectiveness in learning representations on graphs in recent years. To generate ideal augmentation views, the augmentation generation methods should preserve essential information while discarding less relevant details for downstream tasks. However, current augmentation methods usually involve random topology corruption in the spatial domain, which fails to adequately address information spread across different frequencies in the spectral domain. Our preliminary study highlights this issue, demonstrating that spatial random perturbations impact all frequency bands almost uniformly. Given that task-relevant information typically resides in specific spectral regions that vary across graphs, this one-size-fits-all approach can pose challenges. We argue that indiscriminate spatial random perturbation might unintentionally weaken task-relevant information, reducing its effectiveness. To tackle this challenge, we propose applying perturbations selectively, focusing on information specific to different frequencies across diverse graphs. In this paper, we present GASSER, a model that applies tailored perturbations to specific frequencies of graph structures in the spectral domain, guided by spectral hints. Through extensive experimentation and theoretical analysis, we demonstrate that the augmentation views generated by GASSER are adaptive, controllable, and intuitively aligned with the homophily ratios and spectrum of graph structures.

Related papers

• GrassNet: State Space Model Meets Graph Neural Network [57.62885438406724]
  Graph State Space Network (GrassNet) is a novel graph neural network with theoretical support that provides a simple yet effective scheme for designing arbitrary graph spectral filters. To the best of our knowledge, our work is the first to employ SSMs for the design of graph GNN spectral filters. Extensive experiments on nine public benchmarks reveal that GrassNet achieves superior performance in real-world graph modeling tasks.
  arXiv  Detail & Related papers  (2024-08-16T07:33:58Z)
• Spectral Graph Reasoning Network for Hyperspectral Image Classification [0.43512163406551996]
  Convolutional neural networks (CNNs) have achieved remarkable performance in hyperspectral image (HSI) classification. We propose a spectral graph reasoning network (SGR) learning framework comprising two crucial modules. Experiments on two HSI datasets demonstrate that the proposed architecture can significantly improve the classification accuracy.
  arXiv  Detail & Related papers  (2024-07-02T20:29:23Z)
• Heterogeneous Graph Contrastive Learning with Spectral Augmentation [15.231689595121553]
  This paper introduces a spectral-enhanced graph contrastive learning model (SHCL) for the first time in heterogeneous graph neural networks. The proposed model learns an adaptive topology augmentation scheme through the heterogeneous graph itself. Experimental results on multiple real-world datasets demonstrate substantial advantages of the proposed model.
  arXiv  Detail & Related papers  (2024-06-30T14:20:12Z)
• Through the Dual-Prism: A Spectral Perspective on Graph Data Augmentation for Graph Classification [71.36575018271405]
  We introduce the Dual-Prism (DP) augmentation method, comprising DP-Noise and DP-Mask. We find that keeping the low-frequency eigenvalues unchanged can preserve the critical properties at a large scale when generating augmented graphs.
  arXiv  Detail & Related papers  (2024-01-18T12:58:53Z)
• HoloNets: Spectral Convolutions do extend to Directed Graphs [59.851175771106625]
  Conventional wisdom dictates that spectral convolutional networks may only be deployed on undirected graphs. Here we show this traditional reliance on the graph Fourier transform to be superfluous. We provide a frequency-response interpretation of newly developed filters, investigate the influence of the basis used to express filters and discuss the interplay with characteristic operators on which networks are based.
  arXiv  Detail & Related papers  (2023-10-03T17:42:09Z)
• Spectral Augmentation for Self-Supervised Learning on Graphs [43.19199994575821]
  Graph contrastive learning (GCL) aims to learn representations via instance discrimination. It relies on graph augmentation to reflect invariant patterns that are robust to small perturbations. Recent studies mainly perform topology augmentations in a uniformly random manner in the spatial domain. We develop spectral augmentation which guides topology augmentations by maximizing the spectral change.
  arXiv  Detail & Related papers  (2022-10-02T22:20:07Z)
• Quasi-Framelets: Robust Graph Neural Networks via Adaptive Framelet Convolution [28.474359021962346]
  We propose a multiscale framelet convolution for spectral graph neural networks (GNNs) The proposed design excels in filtering out unwanted spectral information and significantly reduces the adverse effects of noisy graph signals. It exhibits remarkable resilience to noisy data and adversarial attacks, highlighting its potential as a robust solution for real-world graph applications.
  arXiv  Detail & Related papers  (2022-01-11T00:10:28Z)
• On the Frequency Bias of Generative Models [61.60834513380388]
  We analyze proposed measures against high-frequency artifacts in state-of-the-art GAN training. We find that none of the existing approaches can fully resolve spectral artifacts yet. Our results suggest that there is great potential in improving the discriminator.
  arXiv  Detail & Related papers  (2021-11-03T18:12:11Z)
• Graph Structural Attack by Spectral Distance [35.998704625736394]
  Graph Convolutional Networks (GCNs) have fueled a surge of interest due to their superior performance on graph learning tasks. In this paper, an effective graph structural attack is investigated to disrupt graph spectral filters in the Fourier domain.
  arXiv  Detail & Related papers  (2021-11-01T04:02:34Z)
• Spectral-Spatial Global Graph Reasoning for Hyperspectral Image Classification [50.899576891296235]
  Convolutional neural networks have been widely applied to hyperspectral image classification. Recent methods attempt to address this issue by performing graph convolutions on spatial topologies.
  arXiv  Detail & Related papers  (2021-06-26T06:24:51Z)
• Spectral Graph Attention Network with Fast Eigen-approximation [103.93113062682633]
  Spectral Graph Attention Network (SpGAT) learns representations for different frequency components regarding weighted filters and graph wavelets bases. Fast approximation variant SpGAT-Cheby is proposed to reduce the computational cost brought by the eigen-decomposition. We thoroughly evaluate the performance of SpGAT and SpGAT-Cheby in semi-supervised node classification tasks.
  arXiv  Detail & Related papers  (2020-03-16T21:49:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.