Simplifying Subgraph Representation Learning for Scalable Link
Prediction
- URL: http://arxiv.org/abs/2301.12562v3
- Date: Wed, 13 Dec 2023 19:15:21 GMT
- Title: Simplifying Subgraph Representation Learning for Scalable Link
Prediction
- Authors: Paul Louis, Shweta Ann Jacob and Amirali Salehi-Abari
- Abstract summary: Subgraph representation learning approaches (SGRLs) transform link prediction to graph classification on the subgraphs around the links.
SGRLs are computationally expensive, and not scalable to large-scale graphs due to expensive subgraph-level operations.
We propose a new class of SGRLs, that we call Scalable Simplified SGRL (S3GRL)
S3GRL simplifies the message passing and aggregation operations in each link's subgraph.
- Score: 2.5782420501870296
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Link prediction on graphs is a fundamental problem. Subgraph representation
learning approaches (SGRLs), by transforming link prediction to graph
classification on the subgraphs around the links, have achieved
state-of-the-art performance in link prediction. However, SGRLs are
computationally expensive, and not scalable to large-scale graphs due to
expensive subgraph-level operations. To unlock the scalability of SGRLs, we
propose a new class of SGRLs, that we call Scalable Simplified SGRL (S3GRL).
Aimed at faster training and inference, S3GRL simplifies the message passing
and aggregation operations in each link's subgraph. S3GRL, as a scalability
framework, accommodates various subgraph sampling strategies and diffusion
operators to emulate computationally-expensive SGRLs. We propose multiple
instances of S3GRL and empirically study them on small to large-scale graphs.
Our extensive experiments demonstrate that the proposed S3GRL models scale up
SGRLs without significant performance compromise (even with considerable gains
in some cases), while offering substantially lower computational footprints
(e.g., multi-fold inference and training speedup).
Related papers
- GES: Generalized Exponential Splatting for Efficient Radiance Field Rendering [112.16239342037714]
GES (Generalized Exponential Splatting) is a novel representation that employs Generalized Exponential Function (GEF) to model 3D scenes.
With the aid of a frequency-modulated loss, GES achieves competitive performance in novel-view synthesis benchmarks.
arXiv Detail & Related papers (2024-02-15T17:32:50Z) - Graph Transformers for Large Graphs [57.19338459218758]
This work advances representation learning on single large-scale graphs with a focus on identifying model characteristics and critical design constraints.
A key innovation of this work lies in the creation of a fast neighborhood sampling technique coupled with a local attention mechanism.
We report a 3x speedup and 16.8% performance gain on ogbn-products and snap-patents, while we also scale LargeGT on ogbn-100M with a 5.9% performance improvement.
arXiv Detail & Related papers (2023-12-18T11:19:23Z) - Rethinking and Simplifying Bootstrapped Graph Latents [48.76934123429186]
Graph contrastive learning (GCL) has emerged as a representative paradigm in graph self-supervised learning.
We present SGCL, a simple yet effective GCL framework that utilizes the outputs from two consecutive iterations as positive pairs.
We show that SGCL can achieve competitive performance with fewer parameters, lower time and space costs, and significant convergence speedup.
arXiv Detail & Related papers (2023-12-05T09:49:50Z) - Efficient Heterogeneous Graph Learning via Random Projection [65.65132884606072]
Heterogeneous Graph Neural Networks (HGNNs) are powerful tools for deep learning on heterogeneous graphs.
Recent pre-computation-based HGNNs use one-time message passing to transform a heterogeneous graph into regular-shaped tensors.
We propose a hybrid pre-computation-based HGNN, named Random Projection Heterogeneous Graph Neural Network (RpHGNN)
arXiv Detail & Related papers (2023-10-23T01:25:44Z) - RLIPv2: Fast Scaling of Relational Language-Image Pre-training [53.21796397618875]
We propose RLIPv2, a fast converging model that enables the relational scaling of pre-training to large-scale pseudo-labelled scene graph data.
Asymmetric Language-Image Fusion (ALIF) facilitates earlier and deeper gated cross-modal fusion with sparsified language encoding.
RLIPv2 shows state-of-the-art performance on three benchmarks under fully-finetuning, few-shot and zero-shot settings.
arXiv Detail & Related papers (2023-08-18T07:17:09Z) - A Comprehensive Study on Large-Scale Graph Training: Benchmarking and
Rethinking [124.21408098724551]
Large-scale graph training is a notoriously challenging problem for graph neural networks (GNNs)
We present a new ensembling training manner, named EnGCN, to address the existing issues.
Our proposed method has achieved new state-of-the-art (SOTA) performance on large-scale datasets.
arXiv Detail & Related papers (2022-10-14T03:43:05Z) - Algorithm and System Co-design for Efficient Subgraph-based Graph
Representation Learning [16.170895692951]
Subgraph-based graph representation learning (SGRL) has been recently proposed to deal with some fundamental challenges encountered by canonical graph neural networks (GNNs)
We propose a novel framework SUREL for scalable SGRL by co-designing the learning algorithm and its system support.
arXiv Detail & Related papers (2022-02-28T04:29:22Z) - Distributed Multi-Agent Reinforcement Learning Based on Graph-Induced Local Value Functions [7.6860514640178]
We propose a general computationally efficient distributed framework for cooperative multi-agent reinforcement learning (MARL)
We introduce three coupling graphs describing three types of inter-agent couplings in MARL.
We propose two distributed RL approaches based on local value-functions derived from the coupling graphs.
arXiv Detail & Related papers (2022-02-26T03:01:51Z) - GIST: Distributed Training for Large-Scale Graph Convolutional Networks [18.964079367668262]
GIST is a hybrid layer and graph sampling method, which disjointly partitions the global model into several, smaller sub-GCNs.
This distributed framework improves model performance and significantly decreases wall-clock training time.
GIST seeks to enable large-scale GCN experimentation with the goal of bridging the existing gap in scale between graph machine learning and deep learning.
arXiv Detail & Related papers (2021-02-20T19:25:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.