Why Propagate Alone? Parallel Use of Labels and Features on Graphs

• URL: http://arxiv.org/abs/2110.07190v1
• Date: Thu, 14 Oct 2021 07:34:11 GMT
• Title: Why Propagate Alone? Parallel Use of Labels and Features on Graphs
• Authors: Yangkun Wang, Jiarui Jin, Weinan Zhang, Yongyi Yang, Jiuhai Chen, Quan Gan, Yong Yu, Zheng Zhang, Zengfeng Huang and David Wipf
• Abstract summary: Graph neural networks (GNNs) and label propagation represent two interrelated modeling strategies designed to exploit graph structure in tasks such as node property prediction. We show that a label trick can be reduced to an interpretable, deterministic training objective composed of two factors.
• Score: 42.01561812621306
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Graph neural networks (GNNs) and label propagation represent two interrelated modeling strategies designed to exploit graph structure in tasks such as node property prediction. The former is typically based on stacked message-passing layers that share neighborhood information to transform node features into predictive embeddings. In contrast, the latter involves spreading label information to unlabeled nodes via a parameter-free diffusion process, but operates independently of the node features. Given then that the material difference is merely whether features or labels are smoothed across the graph, it is natural to consider combinations of the two for improving performance. In this regard, it has recently been proposed to use a randomly-selected portion of the training labels as GNN inputs, concatenated with the original node features for making predictions on the remaining labels. This so-called label trick accommodates the parallel use of features and labels, and is foundational to many of the top-ranking submissions on the Open Graph Benchmark (OGB) leaderboard. And yet despite its wide-spread adoption, thus far there has been little attempt to carefully unpack exactly what statistical properties the label trick introduces into the training pipeline, intended or otherwise. To this end, we prove that under certain simplifying assumptions, the stochastic label trick can be reduced to an interpretable, deterministic training objective composed of two factors. The first is a data-fitting term that naturally resolves potential label leakage issues, while the second serves as a regularization factor conditioned on graph structure that adapts to graph size and connectivity. Later, we leverage this perspective to motivate a broader range of label trick use cases, and provide experiments to verify the efficacy of these extensions.

Related papers

• NP$^2$L: Negative Pseudo Partial Labels Extraction for Graph Neural Networks [48.39834063008816]
  Pseudo labels are used in graph neural networks (GNNs) to assist learning at the message-passing level. In this paper, we introduce a new method to use pseudo labels in GNNs. We show that our method is more accurate if they are selected by not overlapping partial labels and defined as negative node pairs relations.
  arXiv  Detail & Related papers  (2023-10-02T11:13:59Z)
• Label-Enhanced Graph Neural Network for Semi-supervised Node Classification [32.64730237473914]
  We present a label-enhanced learning framework for Graph Neural Networks (GNNs) It first models each label as a virtual center for intra-class nodes and then jointly learns the representations of both nodes and labels. Our approach could not only smooth the representations of nodes belonging to the same class, but also explicitly encode the label semantics into the learning process of GNNs.
  arXiv  Detail & Related papers  (2022-05-31T09:48:47Z)
• Effective Token Graph Modeling using a Novel Labeling Strategy for Structured Sentiment Analysis [39.770652220521384]
  State-of-the-art model for structured sentiment analysis casts the task as a dependency parsing problem. Label proportions for span prediction and span relation prediction are imbalanced. Two nodes in a dependency graph cannot have multiple arcs, therefore some overlapped sentiments cannot be recognized.
  arXiv  Detail & Related papers  (2022-03-21T08:23:03Z)
• Graph Attention Transformer Network for Multi-Label Image Classification [50.0297353509294]
  We propose a general framework for multi-label image classification that can effectively mine complex inter-label relationships. Our proposed methods can achieve state-of-the-art performance on three datasets.
  arXiv  Detail & Related papers  (2022-03-08T12:39:05Z)
• Neural Graph Matching for Pre-training Graph Neural Networks [72.32801428070749]
  Graph neural networks (GNNs) have been shown powerful capacity at modeling structural data. We present a novel Graph Matching based GNN Pre-Training framework, called GMPT. The proposed method can be applied to fully self-supervised pre-training and coarse-grained supervised pre-training.
  arXiv  Detail & Related papers  (2022-03-03T09:53:53Z)
• Label-Wise Message Passing Graph Neural Network on Heterophilic Graphs [20.470934944907608]
  We investigate a novel framework that performs well on graphs with either homophily or heterophily. In label-wise message-passing, neighbors with similar pseudo labels will be aggregated together. We also propose a bi-level optimization method to automatically select the model for graphs with homophily/heterophily.
  arXiv  Detail & Related papers  (2021-10-15T14:49:45Z)
• Adaptive Label Smoothing To Regularize Large-Scale Graph Training [46.00927775402987]
  We propose the adaptive label smoothing (ALS) method to replace the one-hot hard labels with smoothed ones. ALS propagates node labels to aggregate the neighborhood label distribution in a pre-processing step, and then updates the optimal smoothed labels online to adapt to specific graph structure.
  arXiv  Detail & Related papers  (2021-08-30T23:51:31Z)
• Inverse Graph Identification: Can We Identify Node Labels Given Graph Labels? [89.13567439679709]
  Graph Identification (GI) has long been researched in graph learning and is essential in certain applications. This paper defines a novel problem dubbed Inverse Graph Identification (IGI) We propose a simple yet effective method that makes the node-level message passing process using Graph Attention Network (GAT) under the protocol of GI.
  arXiv  Detail & Related papers  (2020-07-12T12:06:17Z)
• Sequential Graph Convolutional Network for Active Learning [53.99104862192055]
  We propose a novel pool-based Active Learning framework constructed on a sequential Graph Convolution Network (GCN) With a small number of randomly sampled images as seed labelled examples, we learn the parameters of the graph to distinguish labelled vs unlabelled nodes. We exploit these characteristics of GCN to select the unlabelled examples which are sufficiently different from labelled ones.
  arXiv  Detail & Related papers  (2020-06-18T00:55:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.