Distribution Consistency based Self-Training for Graph Neural Networks with Sparse Labels

• URL: http://arxiv.org/abs/2401.10394v1
• Date: Thu, 18 Jan 2024 22:07:48 GMT
• Title: Distribution Consistency based Self-Training for Graph Neural Networks with Sparse Labels
• Authors: Fali Wang, Tianxiang Zhao, Suhang Wang
• Abstract summary: Few-shot node classification poses a significant challenge for Graph Neural Networks (GNNs) Self-training has emerged as a widely popular framework to leverage the abundance of unlabeled data. We propose a novel Distribution-Consistent Graph Self-Training framework to identify pseudo-labeled nodes that are both informative and capable of redeeming the distribution discrepancy.
• Score: 33.89511660654271
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Few-shot node classification poses a significant challenge for Graph Neural Networks (GNNs) due to insufficient supervision and potential distribution shifts between labeled and unlabeled nodes. Self-training has emerged as a widely popular framework to leverage the abundance of unlabeled data, which expands the training set by assigning pseudo-labels to selected unlabeled nodes. Efforts have been made to develop various selection strategies based on confidence, information gain, etc. However, none of these methods takes into account the distribution shift between the training and testing node sets. The pseudo-labeling step may amplify this shift and even introduce new ones, hindering the effectiveness of self-training. Therefore, in this work, we explore the potential of explicitly bridging the distribution shift between the expanded training set and test set during self-training. To this end, we propose a novel Distribution-Consistent Graph Self-Training (DC-GST) framework to identify pseudo-labeled nodes that are both informative and capable of redeeming the distribution discrepancy and formulate it as a differentiable optimization task. A distribution-shift-aware edge predictor is further adopted to augment the graph and increase the model's generalizability in assigning pseudo labels. We evaluate our proposed method on four publicly available benchmark datasets and extensive experiments demonstrate that our framework consistently outperforms state-of-the-art baselines.

Related papers

• ALEX: Towards Effective Graph Transfer Learning with Noisy Labels [11.115297917940829]
  We introduce a novel technique termed Balance Alignment and Information-aware Examination (ALEX) to address the problem of graph transfer learning. ALEX first employs singular value decomposition to generate different views with crucial structural semantics, which help provide robust node representations. Building on this foundation, an adversarial domain discriminator is incorporated for the implicit domain alignment of complex multi-modal distributions.
  arXiv  Detail & Related papers  (2023-09-26T04:59:49Z)
• CONVERT:Contrastive Graph Clustering with Reliable Augmentation [110.46658439733106]
  We propose a novel CONtrastiVe Graph ClustEring network with Reliable AugmenTation (CONVERT) In our method, the data augmentations are processed by the proposed reversible perturb-recover network. To further guarantee the reliability of semantics, a novel semantic loss is presented to constrain the network.
  arXiv  Detail & Related papers  (2023-08-17T13:07:09Z)
• All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation [67.30502812804271]
  Pseudo-labels are widely employed in weakly supervised 3D segmentation tasks where only sparse ground-truth labels are available for learning. We propose a novel learning strategy to regularize the generated pseudo-labels and effectively narrow the gaps between pseudo-labels and model predictions.
  arXiv  Detail & Related papers  (2023-05-25T08:19:31Z)
• Transductive Linear Probing: A Novel Framework for Few-Shot Node Classification [56.17097897754628]
  We show that transductive linear probing with self-supervised graph contrastive pretraining can outperform the state-of-the-art fully supervised meta-learning based methods under the same protocol. We hope this work can shed new light on few-shot node classification problems and foster future research on learning from scarcely labeled instances on graphs.
  arXiv  Detail & Related papers  (2022-12-11T21:10:34Z)
• Neighbour Consistency Guided Pseudo-Label Refinement for Unsupervised Person Re-Identification [80.98291772215154]
  Unsupervised person re-identification (ReID) aims at learning discriminative identity features for person retrieval without any annotations. Recent advances accomplish this task by leveraging clustering-based pseudo labels. We propose a Neighbour Consistency guided Pseudo Label Refinement framework.
  arXiv  Detail & Related papers  (2022-11-30T09:39:57Z)
• Similarity-aware Positive Instance Sampling for Graph Contrastive Pre-training [82.68805025636165]
  We propose to select positive graph instances directly from existing graphs in the training set. Our selection is based on certain domain-specific pair-wise similarity measurements. Besides, we develop an adaptive node-level pre-training method to dynamically mask nodes to distribute them evenly in the graph.
  arXiv  Detail & Related papers  (2022-06-23T20:12:51Z)
• Confidence May Cheat: Self-Training on Graph Neural Networks under Distribution Shift [39.73304203101909]
  Self-training methods have been widely adopted on graphs by labeling high-confidence unlabeled nodes and then adding them to the training step. We propose a novel Distribution Recovered Graph Self-Training framework (DR- GST), which could recover the distribution of the original labeled dataset. Both our theoretical analysis and extensive experiments on five benchmark datasets demonstrate the effectiveness of the proposed DR- GST.
  arXiv  Detail & Related papers  (2022-01-27T07:12:27Z)
• Informative Pseudo-Labeling for Graph Neural Networks with Few Labels [12.83841767562179]
  Graph Neural Networks (GNNs) have achieved state-of-the-art results for semi-supervised node classification on graphs. The challenge of how to effectively learn GNNs with very few labels is still under-explored. We propose a novel informative pseudo-labeling framework, called InfoGNN, to facilitate learning of GNNs with extremely few labels.
  arXiv  Detail & Related papers  (2022-01-20T01:49:30Z)
• Scalable and Adaptive Graph Neural Networks with Self-Label-Enhanced training [1.2183405753834562]
  It is hard to directly implement Graph Neural Networks (GNNs) on large scaled graphs. We propose scalable and Adaptive Graph Neural Networks (SAGN) We propose Self-Label-Enhance (SLE) framework combining self-training approach and label propagation in depth.
  arXiv  Detail & Related papers  (2021-04-19T15:08:06Z)
• PseudoSeg: Designing Pseudo Labels for Semantic Segmentation [78.35515004654553]
  We present a re-design of pseudo-labeling to generate structured pseudo labels for training with unlabeled or weakly-labeled data. We demonstrate the effectiveness of the proposed pseudo-labeling strategy in both low-data and high-data regimes.
  arXiv  Detail & Related papers  (2020-10-19T17:59:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.