Augment with Care: Enhancing Graph Contrastive Learning with Selective Spectrum Perturbation

• URL: http://arxiv.org/abs/2310.13845v1
• Date: Fri, 20 Oct 2023 22:39:07 GMT
• Title: Augment with Care: Enhancing Graph Contrastive Learning with Selective Spectrum Perturbation
• Authors: Kaiqi Yang, Haoyu Han, Wei Jin, Hui Liu
• Abstract summary: Graph Contrastive Learning (GCL) has shown remarkable effectiveness in learning representations on graphs. Existing augmentation views with perturbed graph structures are usually based on random topology corruption in the spatial domain. We propose GASSER which poses tailored perturbation on the specific frequencies of graph structures in spectral domain.
• Score: 11.322569167679633
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In recent years, Graph Contrastive Learning (GCL) has shown remarkable effectiveness in learning representations on graphs. As a component of GCL, good augmentation views are supposed to be invariant to the important information while discarding the unimportant part. Existing augmentation views with perturbed graph structures are usually based on random topology corruption in the spatial domain; however, from perspectives of the spectral domain, this approach may be ineffective as it fails to pose tailored impacts on the information of different frequencies, thus weakening the agreement between the augmentation views. By a preliminary experiment, we show that the impacts caused by spatial random perturbation are approximately evenly distributed among frequency bands, which may harm the invariance of augmentations required by contrastive learning frameworks. To address this issue, we argue that the perturbation should be selectively posed on the information concerning different frequencies. In this paper, we propose GASSER which poses tailored perturbation on the specific frequencies of graph structures in spectral domain, and the edge perturbation is selectively guided by the spectral hints. As shown by extensive experiments and theoretical analysis, the augmentation views are adaptive and controllable, as well as heuristically fitting the homophily ratios and spectrum of graph structures.

Related papers

• Spectral Graph Reasoning Network for Hyperspectral Image Classification [0.43512163406551996]
  Convolutional neural networks (CNNs) have achieved remarkable performance in hyperspectral image (HSI) classification. We propose a spectral graph reasoning network (SGR) learning framework comprising two crucial modules. Experiments on two HSI datasets demonstrate that the proposed architecture can significantly improve the classification accuracy.
  arXiv  Detail & Related papers  (2024-07-02T20:29:23Z)
• Through the Dual-Prism: A Spectral Perspective on Graph Data Augmentation for Graph Classification [71.36575018271405]
  We introduce the Dual-Prism (DP) augmentation method, comprising DP-Noise and DP-Mask. We find that keeping the low-frequency eigenvalues unchanged can preserve the critical properties at a large scale when generating augmented graphs.
  arXiv  Detail & Related papers  (2024-01-18T12:58:53Z)
• Shape-aware Graph Spectral Learning [36.63516222161871]
  Spectral Graph Neural Networks (GNNs) are gaining attention for their ability to surpass the limitations of message-passing GNNs. Some works empirically show that the preferred graph frequency is related to the graph homophily level. This relationship between graph frequency and graphs with homophily/heterophily has not been systematically analyzed and considered in existing spectral GNNs. We propose shape-aware regularization on a Newton Interpolation-based spectral filter that can learn an arbitrary spectral filter and incorporate prior knowledge about the desired shape of the corresponding homophily level.
  arXiv  Detail & Related papers  (2023-10-16T04:57:30Z)
• HoloNets: Spectral Convolutions do extend to Directed Graphs [59.851175771106625]
  Conventional wisdom dictates that spectral convolutional networks may only be deployed on undirected graphs. Here we show this traditional reliance on the graph Fourier transform to be superfluous. We provide a frequency-response interpretation of newly developed filters, investigate the influence of the basis used to express filters and discuss the interplay with characteristic operators on which networks are based.
  arXiv  Detail & Related papers  (2023-10-03T17:42:09Z)
• Spectral Augmentations for Graph Contrastive Learning [50.149996923976836]
  Contrastive learning has emerged as a premier method for learning representations with or without supervision. Recent studies have shown its utility in graph representation learning for pre-training. We propose a set of well-motivated graph transformation operations to provide a bank of candidates when constructing augmentations for a graph contrastive objective.
  arXiv  Detail & Related papers  (2023-02-06T16:26:29Z)
• Spectral Feature Augmentation for Graph Contrastive Learning and Beyond [64.78221638149276]
  We present a novel spectral feature argumentation for contrastive learning on graphs (and images) For each data view, we estimate a low-rank approximation per feature map and subtract that approximation from the map to obtain its complement. This is achieved by the proposed herein incomplete power iteration, a non-standard power regime which enjoys two valuable byproducts (under mere one or two iterations) Experiments on graph/image datasets show that our spectral feature augmentation outperforms baselines.
  arXiv  Detail & Related papers  (2022-12-02T08:48:11Z)
• Revisiting Graph Contrastive Learning from the Perspective of Graph Spectrum [91.06367395889514]
  Graph Contrastive Learning (GCL) learning the node representations by augmenting graphs has attracted considerable attentions. We answer these questions by establishing the connection between GCL and graph spectrum. We propose a spectral graph contrastive learning module (SpCo), which is a general and GCL-friendly plug-in.
  arXiv  Detail & Related papers  (2022-10-05T15:32:00Z)
• Spectral Augmentation for Self-Supervised Learning on Graphs [43.19199994575821]
  Graph contrastive learning (GCL) aims to learn representations via instance discrimination. It relies on graph augmentation to reflect invariant patterns that are robust to small perturbations. Recent studies mainly perform topology augmentations in a uniformly random manner in the spatial domain. We develop spectral augmentation which guides topology augmentations by maximizing the spectral change.
  arXiv  Detail & Related papers  (2022-10-02T22:20:07Z)
• Effect Identification in Cluster Causal Diagrams [51.42809552422494]
  We introduce a new type of graphical model called cluster causal diagrams (for short, C-DAGs) C-DAGs allow for the partial specification of relationships among variables based on limited prior knowledge. We develop the foundations and machinery for valid causal inferences over C-DAGs.
  arXiv  Detail & Related papers  (2022-02-22T21:27:31Z)
• Graph Structural Attack by Spectral Distance [35.998704625736394]
  Graph Convolutional Networks (GCNs) have fueled a surge of interest due to their superior performance on graph learning tasks. In this paper, an effective graph structural attack is investigated to disrupt graph spectral filters in the Fourier domain.
  arXiv  Detail & Related papers  (2021-11-01T04:02:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.