Informative Pseudo-Labeling for Graph Neural Networks with Few Labels

• URL: http://arxiv.org/abs/2201.07951v1
• Date: Thu, 20 Jan 2022 01:49:30 GMT
• Title: Informative Pseudo-Labeling for Graph Neural Networks with Few Labels
• Authors: Yayong Li, Jie Yin, Ling Chen
• Abstract summary: Graph Neural Networks (GNNs) have achieved state-of-the-art results for semi-supervised node classification on graphs. The challenge of how to effectively learn GNNs with very few labels is still under-explored. We propose a novel informative pseudo-labeling framework, called InfoGNN, to facilitate learning of GNNs with extremely few labels.
• Score: 12.83841767562179
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Graph Neural Networks (GNNs) have achieved state-of-the-art results for semi-supervised node classification on graphs. Nevertheless, the challenge of how to effectively learn GNNs with very few labels is still under-explored. As one of the prevalent semi-supervised methods, pseudo-labeling has been proposed to explicitly address the label scarcity problem. It aims to augment the training set with pseudo-labeled unlabeled nodes with high confidence so as to re-train a supervised model in a self-training cycle. However, the existing pseudo-labeling approaches often suffer from two major drawbacks. First, they tend to conservatively expand the label set by selecting only high-confidence unlabeled nodes without assessing their informativeness. Unfortunately, those high-confidence nodes often convey overlapping information with given labels, leading to minor improvements for model re-training. Second, these methods incorporate pseudo-labels to the same loss function with genuine labels, ignoring their distinct contributions to the classification task. In this paper, we propose a novel informative pseudo-labeling framework, called InfoGNN, to facilitate learning of GNNs with extremely few labels. Our key idea is to pseudo label the most informative nodes that can maximally represent the local neighborhoods via mutual information maximization. To mitigate the potential label noise and class-imbalance problem arising from pseudo labeling, we also carefully devise a generalized cross entropy loss with a class-balanced regularization to incorporate generated pseudo labels into model re-training. Extensive experiments on six real-world graph datasets demonstrate that our proposed approach significantly outperforms state-of-the-art baselines and strong self-supervised methods on graphs.

Related papers

• Graph Neural Networks with Coarse- and Fine-Grained Division for Mitigating Label Sparsity and Noise [5.943641527857957]
  Graph Neural Networks (GNNs) have gained prominence in semi-supervised learning tasks in processing graph-structured data. In real-world scenarios, labels on nodes of graphs are inevitably noisy and sparsely labeled, significantly degrading the performance of GNNs. We propose a novel GNN-CFGD that reduces the negative impact of noisy labels via coarse- and fine-grained division, along with graph reconstruction.
  arXiv  Detail & Related papers  (2024-11-06T08:21:26Z)
• ERASE: Error-Resilient Representation Learning on Graphs for Label Noise Tolerance [53.73316938815873]
  We propose a method called ERASE (Error-Resilient representation learning on graphs for lAbel noiSe tolerancE) to learn representations with error tolerance. ERASE combines prototype pseudo-labels with propagated denoised labels and updates representations with error resilience. Our method can outperform multiple baselines with clear margins in broad noise levels and enjoy great scalability.
  arXiv  Detail & Related papers  (2023-12-13T17:59:07Z)
• Learning on Graphs under Label Noise [5.909452203428086]
  We develop a novel approach dubbed Consistent Graph Neural Network (CGNN) to solve the problem of learning on graphs with label noise. Specifically, we employ graph contrastive learning as a regularization term, which promotes two views of augmented nodes to have consistent representations. To detect noisy labels on the graph, we present a sample selection technique based on the homophily assumption.
  arXiv  Detail & Related papers  (2023-06-14T01:38:01Z)
• All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation [67.30502812804271]
  Pseudo-labels are widely employed in weakly supervised 3D segmentation tasks where only sparse ground-truth labels are available for learning. We propose a novel learning strategy to regularize the generated pseudo-labels and effectively narrow the gaps between pseudo-labels and model predictions.
  arXiv  Detail & Related papers  (2023-05-25T08:19:31Z)
• Pseudo Contrastive Learning for Graph-based Semi-supervised Learning [67.37572762925836]
  Pseudo Labeling is a technique used to improve the performance of Graph Neural Networks (GNNs) We propose a general framework for GNNs, termed Pseudo Contrastive Learning (PCL)
  arXiv  Detail & Related papers  (2023-02-19T10:34:08Z)
• Label-Enhanced Graph Neural Network for Semi-supervised Node Classification [32.64730237473914]
  We present a label-enhanced learning framework for Graph Neural Networks (GNNs) It first models each label as a virtual center for intra-class nodes and then jointly learns the representations of both nodes and labels. Our approach could not only smooth the representations of nodes belonging to the same class, but also explicitly encode the label semantics into the learning process of GNNs.
  arXiv  Detail & Related papers  (2022-05-31T09:48:47Z)
• NRGNN: Learning a Label Noise-Resistant Graph Neural Network on Sparsely and Noisily Labeled Graphs [20.470934944907608]
  Graph Neural Networks (GNNs) have achieved promising results for semi-supervised learning tasks on graphs such as node classification. Many real-world graphs are often sparsely and noisily labeled, which could significantly degrade the performance of GNNs. We propose to develop a label noise-resistant GNN for semi-supervised node classification.
  arXiv  Detail & Related papers  (2021-06-08T22:12:44Z)
• Boosting Semi-Supervised Face Recognition with Noise Robustness [54.342992887966616]
  This paper presents an effective solution to semi-supervised face recognition that is robust to the label noise aroused by the auto-labelling. We develop a semi-supervised face recognition solution, named Noise Robust Learning-Labelling (NRoLL), which is based on the robust training ability empowered by GN.
  arXiv  Detail & Related papers  (2021-05-10T14:43:11Z)
• Unified Robust Training for Graph NeuralNetworks against Label Noise [12.014301020294154]
  We propose a new framework, UnionNET, for learning with noisy labels on graphs under a semi-supervised setting. Our approach provides a unified solution for robustly training GNNs and performing label correction simultaneously.
  arXiv  Detail & Related papers  (2021-03-05T01:17:04Z)
• On the Equivalence of Decoupled Graph Convolution Network and Label Propagation [60.34028546202372]
  Some work shows that coupling is inferior to decoupling, which supports deep graph propagation better. Despite effectiveness, the working mechanisms of the decoupled GCN are not well understood. We propose a new label propagation method named propagation then training Adaptively (PTA), which overcomes the flaws of the decoupled GCN.
  arXiv  Detail & Related papers  (2020-10-23T13:57:39Z)
• Learn to Propagate Reliably on Noisy Affinity Graphs [69.97364913330989]
  Recent works have shown that exploiting unlabeled data through label propagation can substantially reduce the labeling cost. How to propagate labels reliably, especially on a dataset with unknown outliers, remains an open question. We propose a new framework that allows labels to be propagated reliably on large-scale real-world data.
  arXiv  Detail & Related papers  (2020-07-17T07:55:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.