Mutual Teaching for Graph Convolutional Networks

• URL: http://arxiv.org/abs/2009.00952v1
• Date: Wed, 2 Sep 2020 11:10:55 GMT
• Title: Mutual Teaching for Graph Convolutional Networks
• Authors: Kun Zhan, Chaoxi Niu
• Abstract summary: Graph convolutional networks produce good predictions of unlabeled samples due to its transductive label propagation. Since samples have different predicted confidences, we take high-confidence predictions as pseudo labels to expand the label set. We propose a new training method named as mutual teaching, i.e., we train dual models and let them teach each other during each batch.
• Score: 1.14219428942199
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph convolutional networks produce good predictions of unlabeled samples due to its transductive label propagation. Since samples have different predicted confidences, we take high-confidence predictions as pseudo labels to expand the label set so that more samples are selected for updating models. We propose a new training method named as mutual teaching, i.e., we train dual models and let them teach each other during each batch. First, each network feeds forward all samples and selects samples with high-confidence predictions. Second, each model is updated by samples selected by its peer network. We view the high-confidence predictions as useful knowledge, and the useful knowledge of one network teaches the peer network with model updating in each batch. In mutual teaching, the pseudo-label set of a network is from its peer network. Since we use the new strategy of network training, performance improves significantly. Extensive experimental results demonstrate that our method achieves superior performance over state-of-the-art methods under very low label rates.

Related papers

• Improving Network Interpretability via Explanation Consistency Evaluation [56.14036428778861]
  We propose a framework that acquires more explainable activation heatmaps and simultaneously increase the model performance. Specifically, our framework introduces a new metric, i.e., explanation consistency, to reweight the training samples adaptively in model learning. Our framework then promotes the model learning by paying closer attention to those training samples with a high difference in explanations.
  arXiv  Detail & Related papers  (2024-08-08T17:20:08Z)
• Distribution Shift Matters for Knowledge Distillation with Webly Collected Images [91.66661969598755]
  We propose a novel method dubbed Knowledge Distillation between Different Distributions" (KD$3$) We first dynamically select useful training instances from the webly collected data according to the combined predictions of teacher network and student network. We also build a new contrastive learning block called MixDistribution to generate perturbed data with a new distribution for instance alignment.
  arXiv  Detail & Related papers  (2023-07-21T10:08:58Z)
• MSVQ: Self-Supervised Learning with Multiple Sample Views and Queues [10.327408694770709]
  We propose a new simple framework, namely Multiple Sample Views and Queues (MSVQ) We jointly construct three soft labels on-the-fly by utilizing two complementary and symmetric approaches. Let the student network mimic the similarity relationships between the samples, thus giving the student network a more flexible ability to identify false negative samples in the dataset.
  arXiv  Detail & Related papers  (2023-05-09T12:05:14Z)
• Compare learning: bi-attention network for few-shot learning [6.559037166322981]
  One of the Few-shot learning methods called metric learning addresses this challenge by first learning a deep distance metric to determine whether a pair of images belong to the same category. In this paper, we propose a novel approach named Bi-attention network to compare the instances, which can measure the similarity between embeddings of instances precisely, globally and efficiently.
  arXiv  Detail & Related papers  (2022-03-25T07:39:10Z)
• Scalable Consistency Training for Graph Neural Networks via Self-Ensemble Self-Distillation [13.815063206114713]
  We introduce a novel consistency training method to improve accuracy of graph neural networks (GNNs) For a target node we generate different neighborhood expansions, and distill the knowledge of the average of the predictions to the GNN. Our method approximates the expected prediction of the possible neighborhood samples and practically only requires a few samples.
  arXiv  Detail & Related papers  (2021-10-12T19:24:42Z)
• When Deep Learners Change Their Mind: Learning Dynamics for Active Learning [32.792098711779424]
  In this paper, we propose a new informativeness-based active learning method. Our measure is derived from the learning dynamics of a neural network. We show that label-dispersion is a promising predictor of the uncertainty of the network.
  arXiv  Detail & Related papers  (2021-07-30T15:30:17Z)
• An Empirical Study of the Collapsing Problem in Semi-Supervised 2D Human Pose Estimation [80.02124918255059]
  Semi-supervised learning aims to boost the accuracy of a model by exploring unlabeled images. We learn two networks to mutually teach each other. The more reliable predictions on easy images in each network are used to teach the other network to learn about the corresponding hard images.
  arXiv  Detail & Related papers  (2020-11-25T03:29:52Z)
• SLADE: A Self-Training Framework For Distance Metric Learning [75.54078592084217]
  We present a self-training framework, SLADE, to improve retrieval performance by leveraging additional unlabeled data. We first train a teacher model on the labeled data and use it to generate pseudo labels for the unlabeled data. We then train a student model on both labels and pseudo labels to generate final feature embeddings.
  arXiv  Detail & Related papers  (2020-11-20T08:26:10Z)
• Graph-Based Neural Network Models with Multiple Self-Supervised Auxiliary Tasks [79.28094304325116]
  Graph Convolutional Networks are among the most promising approaches for capturing relationships among structured data points. We propose three novel self-supervised auxiliary tasks to train graph-based neural network models in a multi-task fashion.
  arXiv  Detail & Related papers  (2020-11-14T11:09:51Z)
• DivideMix: Learning with Noisy Labels as Semi-supervised Learning [111.03364864022261]
  We propose DivideMix, a framework for learning with noisy labels. Experiments on multiple benchmark datasets demonstrate substantial improvements over state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-02-18T06:20:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.