ImGCL: Revisiting Graph Contrastive Learning on Imbalanced Node Classification

• URL: http://arxiv.org/abs/2205.11332v2
• Date: Wed, 3 May 2023 06:00:40 GMT
• Title: ImGCL: Revisiting Graph Contrastive Learning on Imbalanced Node Classification
• Authors: Liang Zeng, Lanqing Li, Ziqi Gao, Peilin Zhao, Jian Li
• Abstract summary: Graph contrastive learning (GCL) has attracted a surge of attention due to its superior performance for learning node/graph representations without labels. In practice, the underlying class distribution of unlabeled nodes for the given graph is usually imbalanced. We propose a principled GCL framework on Imbalanced node classification (ImGCL), which automatically and adaptively balances the representations learned from GCL without labels.
• Score: 26.0350727426613
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph contrastive learning (GCL) has attracted a surge of attention due to its superior performance for learning node/graph representations without labels. However, in practice, the underlying class distribution of unlabeled nodes for the given graph is usually imbalanced. This highly imbalanced class distribution inevitably deteriorates the quality of learned node representations in GCL. Indeed, we empirically find that most state-of-the-art GCL methods cannot obtain discriminative representations and exhibit poor performance on imbalanced node classification. Motivated by this observation, we propose a principled GCL framework on Imbalanced node classification (ImGCL), which automatically and adaptively balances the representations learned from GCL without labels. Specifically, we first introduce the online clustering based progressively balanced sampling (PBS) method with theoretical rationale, which balances the training sets based on pseudo-labels obtained from learned representations in GCL. We then develop the node centrality based PBS method to better preserve the intrinsic structure of graphs, by upweighting the important nodes of the given graph. Extensive experiments on multiple imbalanced graph datasets and imbalanced settings demonstrate the effectiveness of our proposed framework, which significantly improves the performance of the recent state-of-the-art GCL methods. Further experimental ablations and analyses show that the ImGCL framework consistently improves the representation quality of nodes in under-represented (tail) classes.

Related papers

• Rethinking and Simplifying Bootstrapped Graph Latents [48.76934123429186]
  Graph contrastive learning (GCL) has emerged as a representative paradigm in graph self-supervised learning. We present SGCL, a simple yet effective GCL framework that utilizes the outputs from two consecutive iterations as positive pairs. We show that SGCL can achieve competitive performance with fewer parameters, lower time and space costs, and significant convergence speedup.
  arXiv  Detail & Related papers  (2023-12-05T09:49:50Z)
• VIGraph: Generative Self-supervised Learning for Class-Imbalanced Node Classification [9.686218058331061]
  Class imbalance in graph data presents significant challenges for node classification. Existing methods, such as SMOTE-based approaches, exhibit limitations in constructing imbalanced graphs. We introduce VIGraph, a simple yet effective generative SSL approach that relies on the Variational GAE as the fundamental model.
  arXiv  Detail & Related papers  (2023-11-02T12:36:19Z)
• Heterophily-Based Graph Neural Network for Imbalanced Classification [19.51668009720269]
  We introduce a unique approach that tackles imbalanced classification on graphs by considering graph heterophily. We propose Fast Im-GBK, which integrates an imbalance classification strategy with heterophily-aware GNNs. Our experiments on real-world graphs demonstrate our model's superiority in classification performance and efficiency for node classification tasks.
  arXiv  Detail & Related papers  (2023-10-12T21:19:47Z)
• Provable Training for Graph Contrastive Learning [58.8128675529977]
  Graph Contrastive Learning (GCL) has emerged as a popular training approach for learning node embeddings from augmented graphs without labels. We show that the training of GCL is indeed imbalanced across all nodes. We propose the metric "node compactness", which is the lower bound of how a node follows the GCL principle.
  arXiv  Detail & Related papers  (2023-09-25T08:23:53Z)
• HomoGCL: Rethinking Homophily in Graph Contrastive Learning [64.85392028383164]
  HomoGCL is a model-agnostic framework to expand the positive set using neighbor nodes with neighbor-specific significances. We show that HomoGCL yields multiple state-of-the-art results across six public datasets.
  arXiv  Detail & Related papers  (2023-06-16T04:06:52Z)
• Localized Contrastive Learning on Graphs [110.54606263711385]
  We introduce a simple yet effective contrastive model named Localized Graph Contrastive Learning (Local-GCL) In spite of its simplicity, Local-GCL achieves quite competitive performance in self-supervised node representation learning tasks on graphs with various scales and properties.
  arXiv  Detail & Related papers  (2022-12-08T23:36:00Z)
• Single-Pass Contrastive Learning Can Work for Both Homophilic and Heterophilic Graph [60.28340453547902]
  Graph contrastive learning (GCL) techniques typically require two forward passes for a single instance to construct the contrastive loss. Existing GCL approaches fail to provide strong performance guarantees. We implement the Single-Pass Graph Contrastive Learning method (SP-GCL) Empirically, the features learned by the SP-GCL can match or outperform existing strong baselines with significantly less computational overhead.
  arXiv  Detail & Related papers  (2022-11-20T07:18:56Z)
• Uncovering the Structural Fairness in Graph Contrastive Learning [87.65091052291544]
  Graph contrastive learning (GCL) has emerged as a promising self-supervised approach for learning node representations. We show that representations obtained by GCL methods are already fairer to degree bias than those learned by GCN. We devise a novel graph augmentation method, called GRAph contrastive learning for DEgree bias (GRADE), which applies different strategies to low- and high-degree nodes.
  arXiv  Detail & Related papers  (2022-10-06T15:58:25Z)
• Augmentation-Free Graph Contrastive Learning [16.471928573824854]
  Graph contrastive learning (GCL) is the most representative and prevalent self-supervised learning approach for graph-structured data. Existing GCL methods rely on an augmentation scheme to learn the representations invariant across different augmentation views. We propose a novel, theoretically-principled, and augmentation-free GCL, named AF-GCL, that leverages the features aggregated by Graph Neural Network to construct the self-supervision signal instead of augmentations.
  arXiv  Detail & Related papers  (2022-04-11T05:37:03Z)
• GCN-Based Linkage Prediction for Face Clustering on Imbalanced Datasets: An Empirical Study [5.416933126354173]
  We present a new method to alleviate the imbalanced labels and also augment graph representations using a Reverse-Imbalance Weighted Sampling strategy. The code and a series of imbalanced benchmark datasets are available at https://github.com/espectre/GCNs_on_imbalanced_datasets.
  arXiv  Detail & Related papers  (2021-07-06T08:45:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.