Low-Rank Graph Contrastive Learning for Node Classification

• URL: http://arxiv.org/abs/2402.09600v1
• Date: Wed, 14 Feb 2024 22:15:37 GMT
• Title: Low-Rank Graph Contrastive Learning for Node Classification
• Authors: Yancheng Wang, Yingzhen Yang
• Abstract summary: Graph Neural Networks (GNNs) have been widely used to learn node representations and with outstanding performance on various tasks such as node classification. We propose a novel and robust GNN encoder, Low-Rank Graph Contrastive Learning (LR-GCL)
• Score: 10.520101507424577
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph Neural Networks (GNNs) have been widely used to learn node representations and with outstanding performance on various tasks such as node classification. However, noise, which inevitably exists in real-world graph data, would considerably degrade the performance of GNNs revealed by recent studies. In this work, we propose a novel and robust GNN encoder, Low-Rank Graph Contrastive Learning (LR-GCL). Our method performs transductive node classification in two steps. First, a low-rank GCL encoder named LR-GCL is trained by prototypical contrastive learning with low-rank regularization. Next, using the features produced by LR-GCL, a linear transductive classification algorithm is used to classify the unlabeled nodes in the graph. Our LR-GCL is inspired by the low frequency property of the graph data and its labels, and it is also theoretically motivated by our sharp generalization bound for transductive learning. To the best of our knowledge, our theoretical result is among the first to theoretically demonstrate the advantage of low-rank learning in graph contrastive learning supported by strong empirical performance. Extensive experiments on public benchmarks demonstrate the superior performance of LR-GCL and the robustness of the learned node representations. The code of LR-GCL is available at \url{https://anonymous.4open.science/r/Low-Rank_Graph_Contrastive_Learning-64A6/}.

Related papers

• GRE^2-MDCL: Graph Representation Embedding Enhanced via Multidimensional Contrastive Learning [0.0]
  Graph representation learning has emerged as a powerful tool for preserving graph topology when mapping nodes to vector representations. Current graph neural network models face the challenge of requiring extensive labeled data. We propose Graph Representation Embedding Enhanced via Multidimensional Contrastive Learning.
  arXiv  Detail & Related papers  (2024-09-12T03:09:05Z)
• Architecture Matters: Uncovering Implicit Mechanisms in Graph Contrastive Learning [34.566003077992384]
  We present a systematic study of various graph contrastive learning (GCL) methods. By uncovering how the implicit inductive bias of GNNs works in contrastive learning, we theoretically provide insights into the above intriguing properties of GCL. Rather than directly porting existing NN methods to GCL, we advocate for more attention toward the unique architecture of graph learning.
  arXiv  Detail & Related papers  (2023-11-05T15:54:17Z)
• Label Deconvolution for Node Representation Learning on Large-scale Attributed Graphs against Learning Bias [75.44877675117749]
  We propose an efficient label regularization technique, namely Label Deconvolution (LD), to alleviate the learning bias by a novel and highly scalable approximation to the inverse mapping of GNNs. Experiments demonstrate LD significantly outperforms state-of-the-art methods on Open Graph datasets Benchmark.
  arXiv  Detail & Related papers  (2023-09-26T13:09:43Z)
• SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning [131.04781590452308]
  We present SimTeG, a frustratingly Simple approach for Textual Graph learning. We first perform supervised parameter-efficient fine-tuning (PEFT) on a pre-trained LM on the downstream task. We then generate node embeddings using the last hidden states of finetuned LM.
  arXiv  Detail & Related papers  (2023-08-03T07:00:04Z)
• LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation [9.181689366185038]
  Graph neural clustering network (GNN) is a powerful learning approach for graph-based recommender systems. In this paper, we propose a simple yet effective graph contrastive learning paradigm LightGCL.
  arXiv  Detail & Related papers  (2023-02-16T10:16:21Z)
• Uncovering the Structural Fairness in Graph Contrastive Learning [87.65091052291544]
  Graph contrastive learning (GCL) has emerged as a promising self-supervised approach for learning node representations. We show that representations obtained by GCL methods are already fairer to degree bias than those learned by GCN. We devise a novel graph augmentation method, called GRAph contrastive learning for DEgree bias (GRADE), which applies different strategies to low- and high-degree nodes.
  arXiv  Detail & Related papers  (2022-10-06T15:58:25Z)
• Let Invariant Rationale Discovery Inspire Graph Contrastive Learning [98.10268114789775]
  We argue that a high-performing augmentation should preserve the salient semantics of anchor graphs regarding instance-discrimination. We propose a new framework, Rationale-aware Graph Contrastive Learning (RGCL) RGCL uses a rationale generator to reveal salient features about graph instance-discrimination as the rationale, and then creates rationale-aware views for contrastive learning.
  arXiv  Detail & Related papers  (2022-06-16T01:28:40Z)
• Bayesian Robust Graph Contrastive Learning [4.6761071607574545]
  We propose a novel and robust method, which trains a GNN encoder to learn robust node representations. Experiments on public and large-scale benchmarks demonstrate the superior performance of BRGCL and the robustness of the learned node representations.
  arXiv  Detail & Related papers  (2022-05-27T17:21:17Z)
• Adversarial Graph Augmentation to Improve Graph Contrastive Learning [21.54343383921459]
  We propose a novel principle, termed adversarial-GCL (AD-GCL), which enables GNNs to avoid capturing redundant information during the training. We experimentally validate AD-GCL by comparing with the state-of-the-art GCL methods and achieve performance gains of up-to $14%$ in unsupervised, $6%$ in transfer, and $3%$ in semi-supervised learning settings.
  arXiv  Detail & Related papers  (2021-06-10T15:34:26Z)
• Combining Label Propagation and Simple Models Out-performs Graph Neural Networks [52.121819834353865]
  We show that for many standard transductive node classification benchmarks, we can exceed or match the performance of state-of-the-art GNNs. We call this overall procedure Correct and Smooth (C&S) Our approach exceeds or nearly matches the performance of state-of-the-art GNNs on a wide variety of benchmarks.
  arXiv  Detail & Related papers  (2020-10-27T02:10:52Z)
• Fast Graph Attention Networks Using Effective Resistance Based Graph Sparsification [70.50751397870972]
  FastGAT is a method to make attention based GNNs lightweight by using spectral sparsification to generate an optimal pruning of the input graph. We experimentally evaluate FastGAT on several large real world graph datasets for node classification tasks.
  arXiv  Detail & Related papers  (2020-06-15T22:07:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.