Efficient Heterogeneous Graph Learning via Random Projection
- URL: http://arxiv.org/abs/2310.14481v1
- Date: Mon, 23 Oct 2023 01:25:44 GMT
- Title: Efficient Heterogeneous Graph Learning via Random Projection
- Authors: Jun Hu, Bryan Hooi, Bingsheng He
- Abstract summary: Heterogeneous Graph Neural Networks (HGNNs) are powerful tools for deep learning on heterogeneous graphs.
Recent pre-computation-based HGNNs use one-time message passing to transform a heterogeneous graph into regular-shaped tensors.
We propose a hybrid pre-computation-based HGNN, named Random Projection Heterogeneous Graph Neural Network (RpHGNN)
- Score: 65.65132884606072
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Heterogeneous Graph Neural Networks (HGNNs) are powerful tools for deep
learning on heterogeneous graphs. Typical HGNNs require repetitive message
passing during training, limiting efficiency for large-scale real-world graphs.
Recent pre-computation-based HGNNs use one-time message passing to transform a
heterogeneous graph into regular-shaped tensors, enabling efficient mini-batch
training. Existing pre-computation-based HGNNs can be mainly categorized into
two styles, which differ in how much information loss is allowed and
efficiency. We propose a hybrid pre-computation-based HGNN, named Random
Projection Heterogeneous Graph Neural Network (RpHGNN), which combines the
benefits of one style's efficiency with the low information loss of the other
style. To achieve efficiency, the main framework of RpHGNN consists of
propagate-then-update iterations, where we introduce a Random Projection
Squashing step to ensure that complexity increases only linearly. To achieve
low information loss, we introduce a Relation-wise Neighbor Collection
component with an Even-odd Propagation Scheme, which aims to collect
information from neighbors in a finer-grained way. Experimental results
indicate that our approach achieves state-of-the-art results on seven small and
large benchmark datasets while also being 230% faster compared to the most
effective baseline. Surprisingly, our approach not only surpasses
pre-processing-based baselines but also outperforms end-to-end methods.
Related papers
- EggNet: An Evolving Graph-based Graph Attention Network for Particle Track Reconstruction [0.0]
We consider a one-shot OC approach that reconstructs particle tracks directly from a set of hits.
This approach iteratively updates the graphs and can better facilitate the message passing across each graph.
Preliminary studies on the TrackML dataset show better track performance compared to the methods that require a fixed input graph.
arXiv Detail & Related papers (2024-07-18T22:29:24Z) - Learning to Reweight for Graph Neural Network [63.978102332612906]
Graph Neural Networks (GNNs) show promising results for graph tasks.
Existing GNNs' generalization ability will degrade when there exist distribution shifts between testing and training graph data.
We propose a novel nonlinear graph decorrelation method, which can substantially improve the out-of-distribution generalization ability.
arXiv Detail & Related papers (2023-12-19T12:25:10Z) - SCARA: Scalable Graph Neural Networks with Feature-Oriented Optimization [23.609017952951454]
We propose SCARA, a scalable Graph Neural Network (GNN) with feature-oriented optimization for graph computation.
SCARA efficiently computes graph embedding from node features, and further selects and reuses feature results to reduce overhead.
It is efficient to process precomputation on the largest available billion-scale GNN dataset Papers100M (111M nodes, 1.6B edges) in 100 seconds.
arXiv Detail & Related papers (2022-07-19T10:32:11Z) - Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural
Networks [52.566735716983956]
We propose a graph gradual pruning framework termed CGP to dynamically prune GNNs.
Unlike LTH-based methods, the proposed CGP approach requires no re-training, which significantly reduces the computation costs.
Our proposed strategy greatly improves both training and inference efficiency while matching or even exceeding the accuracy of existing methods.
arXiv Detail & Related papers (2022-07-18T14:23:31Z) - Optimal Propagation for Graph Neural Networks [51.08426265813481]
We propose a bi-level optimization approach for learning the optimal graph structure.
We also explore a low-rank approximation model for further reducing the time complexity.
arXiv Detail & Related papers (2022-05-06T03:37:00Z) - Neural Graph Matching for Pre-training Graph Neural Networks [72.32801428070749]
Graph neural networks (GNNs) have been shown powerful capacity at modeling structural data.
We present a novel Graph Matching based GNN Pre-Training framework, called GMPT.
The proposed method can be applied to fully self-supervised pre-training and coarse-grained supervised pre-training.
arXiv Detail & Related papers (2022-03-03T09:53:53Z) - Combining Label Propagation and Simple Models Out-performs Graph Neural
Networks [52.121819834353865]
We show that for many standard transductive node classification benchmarks, we can exceed or match the performance of state-of-the-art GNNs.
We call this overall procedure Correct and Smooth (C&S)
Our approach exceeds or nearly matches the performance of state-of-the-art GNNs on a wide variety of benchmarks.
arXiv Detail & Related papers (2020-10-27T02:10:52Z) - Meta-path Free Semi-supervised Learning for Heterogeneous Networks [16.641434334366227]
Graph neural networks (GNNs) have been widely used in representation learning on graphs and achieved superior performance in tasks such as node classification.
In this paper, we propose simple and effective graph neural networks for heterogeneous graph, excluding the use of meta-paths.
arXiv Detail & Related papers (2020-10-18T06:01:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.