Heterogeneous Graph Transformer
- URL: http://arxiv.org/abs/2003.01332v1
- Date: Tue, 3 Mar 2020 04:49:21 GMT
- Title: Heterogeneous Graph Transformer
- Authors: Ziniu Hu, Yuxiao Dong, Kuansan Wang, Yizhou Sun
- Abstract summary: Heterogeneous Graph Transformer (HGT) architecture for modeling Web-scale heterogeneous graphs.
To handle dynamic heterogeneous graphs, we introduce the relative temporal encoding technique into HGT.
To handle Web-scale graph data, we design the heterogeneous mini-batch graph sampling algorithm---HGSampling---for efficient and scalable training.
- Score: 49.675064816860505
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recent years have witnessed the emerging success of graph neural networks
(GNNs) for modeling structured data. However, most GNNs are designed for
homogeneous graphs, in which all nodes and edges belong to the same types,
making them infeasible to represent heterogeneous structures. In this paper, we
present the Heterogeneous Graph Transformer (HGT) architecture for modeling
Web-scale heterogeneous graphs. To model heterogeneity, we design node- and
edge-type dependent parameters to characterize the heterogeneous attention over
each edge, empowering HGT to maintain dedicated representations for different
types of nodes and edges. To handle dynamic heterogeneous graphs, we introduce
the relative temporal encoding technique into HGT, which is able to capture the
dynamic structural dependency with arbitrary durations. To handle Web-scale
graph data, we design the heterogeneous mini-batch graph sampling
algorithm---HGSampling---for efficient and scalable training. Extensive
experiments on the Open Academic Graph of 179 million nodes and 2 billion edges
show that the proposed HGT model consistently outperforms all the
state-of-the-art GNN baselines by 9%--21% on various downstream tasks.
Related papers
- Spectral Greedy Coresets for Graph Neural Networks [61.24300262316091]
The ubiquity of large-scale graphs in node-classification tasks hinders the real-world applications of Graph Neural Networks (GNNs)
This paper studies graph coresets for GNNs and avoids the interdependence issue by selecting ego-graphs based on their spectral embeddings.
Our spectral greedy graph coreset (SGGC) scales to graphs with millions of nodes, obviates the need for model pre-training, and applies to low-homophily graphs.
arXiv Detail & Related papers (2024-05-27T17:52:12Z) - Seq-HGNN: Learning Sequential Node Representation on Heterogeneous Graph [57.2953563124339]
We propose a novel heterogeneous graph neural network with sequential node representation, namely Seq-HGNN.
We conduct extensive experiments on four widely used datasets from Heterogeneous Graph Benchmark (HGB) and Open Graph Benchmark (OGB)
arXiv Detail & Related papers (2023-05-18T07:27:18Z) - Relation Embedding based Graph Neural Networks for Handling
Heterogeneous Graph [58.99478502486377]
We propose a simple yet efficient framework to make the homogeneous GNNs have adequate ability to handle heterogeneous graphs.
Specifically, we propose Relation Embedding based Graph Neural Networks (RE-GNNs), which employ only one parameter per relation to embed the importance of edge type relations and self-loop connections.
arXiv Detail & Related papers (2022-09-23T05:24:18Z) - Descent Steps of a Relation-Aware Energy Produce Heterogeneous Graph
Neural Networks [25.59092732148598]
Heterogeneous graph neural networks (GNNs) achieve strong performance on node classification tasks in a semi-supervised learning setting.
We propose a novel heterogeneous GNN architecture in which layers are derived from optimization steps that descend a novel relation-aware energy function.
arXiv Detail & Related papers (2022-06-22T13:48:08Z) - ES-GNN: Generalizing Graph Neural Networks Beyond Homophily with Edge Splitting [32.69196871253339]
We propose a novel Edge Splitting GNN (ES-GNN) framework to adaptively distinguish between graph edges either relevant or irrelevant to learning tasks.
We show that our ES-GNN can be regarded as a solution to a disentangled graph denoising problem.
arXiv Detail & Related papers (2022-05-27T01:29:03Z) - Incorporating Heterophily into Graph Neural Networks for Graph Classification [6.709862924279403]
Graph Neural Networks (GNNs) often assume strong homophily for graph classification, seldom considering heterophily.
We develop a novel GNN architecture called IHGNN (short for Incorporating Heterophily into Graph Neural Networks)
We empirically validate IHGNN on various graph datasets and demonstrate that it outperforms the state-of-the-art GNNs for graph classification.
arXiv Detail & Related papers (2022-03-15T06:48:35Z) - Graph Neural Networks for Graphs with Heterophily: A Survey [98.45621222357397]
We provide a comprehensive review of graph neural networks (GNNs) for heterophilic graphs.
Specifically, we propose a systematic taxonomy that essentially governs existing heterophilic GNN models.
We discuss the correlation between graph heterophily and various graph research domains, aiming to facilitate the development of more effective GNNs.
arXiv Detail & Related papers (2022-02-14T23:07:47Z) - ACE-HGNN: Adaptive Curvature Exploration Hyperbolic Graph Neural Network [72.16255675586089]
We propose an Adaptive Curvature Exploration Hyperbolic Graph NeuralNetwork named ACE-HGNN to adaptively learn the optimal curvature according to the input graph and downstream tasks.
Experiments on multiple real-world graph datasets demonstrate a significant and consistent performance improvement in model quality with competitive performance and good generalization ability.
arXiv Detail & Related papers (2021-10-15T07:18:57Z) - Meta-path Free Semi-supervised Learning for Heterogeneous Networks [16.641434334366227]
Graph neural networks (GNNs) have been widely used in representation learning on graphs and achieved superior performance in tasks such as node classification.
In this paper, we propose simple and effective graph neural networks for heterogeneous graph, excluding the use of meta-paths.
arXiv Detail & Related papers (2020-10-18T06:01:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.