Accelerating Dynamic Network Embedding with Billions of Parameter Updates to Milliseconds

• URL: http://arxiv.org/abs/2306.08967v1
• Date: Thu, 15 Jun 2023 09:02:17 GMT
• Title: Accelerating Dynamic Network Embedding with Billions of Parameter Updates to Milliseconds
• Authors: Haoran Deng, Yang Yang, Jiahe Li, Haoyang Cai, Shiliang Pu, Weihao Jiang
• Abstract summary: We propose a novel dynamic network embedding paradigm that rotates and scales the axes of embedding space instead of a node-by-node update. Specifically, we propose the Dynamic Adjacency Matrix Factorization (DAMF) algorithm, which achieves an efficient and accurate dynamic network embedding. Experiments of node classification, link prediction, and graph reconstruction on different-sized dynamic graphs suggest that DAMF advances dynamic network embedding.
• Score: 27.98359191399847
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Network embedding, a graph representation learning method illustrating network topology by mapping nodes into lower-dimension vectors, is challenging to accommodate the ever-changing dynamic graphs in practice. Existing research is mainly based on node-by-node embedding modifications, which falls into the dilemma of efficient calculation and accuracy. Observing that the embedding dimensions are usually much smaller than the number of nodes, we break this dilemma with a novel dynamic network embedding paradigm that rotates and scales the axes of embedding space instead of a node-by-node update. Specifically, we propose the Dynamic Adjacency Matrix Factorization (DAMF) algorithm, which achieves an efficient and accurate dynamic network embedding by rotating and scaling the coordinate system where the network embedding resides with no more than the number of edge modifications changes of node embeddings. Moreover, a dynamic Personalized PageRank is applied to the obtained network embeddings to enhance node embeddings and capture higher-order neighbor information dynamically. Experiments of node classification, link prediction, and graph reconstruction on different-sized dynamic graphs suggest that DAMF advances dynamic network embedding. Further, we unprecedentedly expand dynamic network embedding experiments to billion-edge graphs, where DAMF updates billion-level parameters in less than 10ms.

Related papers

• Discovering Message Passing Hierarchies for Mesh-Based Physics Simulation [61.89682310797067]
  We introduce DHMP, which learns Dynamic Hierarchies for Message Passing networks through a differentiable node selection method. Our experiments demonstrate the effectiveness of DHMP, achieving 22.7% improvement on average compared to recent fixed-hierarchy message passing networks.
  arXiv  Detail & Related papers  (2024-10-03T15:18:00Z)
• Online Learning Of Expanding Graphs [14.952056744888916]
  This paper addresses the problem of online network inference for expanding graphs from a stream of signals. We introduce a strategy that enables different types of updates for nodes that just joined the network and for previously existing nodes.
  arXiv  Detail & Related papers  (2024-09-13T09:20:42Z)
• DYMOND: DYnamic MOtif-NoDes Network Generative Model [14.207034497262741]
  We propose DYnamic MOtif-NoDes -- a generative model that considers changes in overall graph structure using temporal motif activity. We show that DYMOND performs better at generating graph structure and node behavior similar to the observed network. We also propose a new methodology to adapt graph structure metrics to better evaluate the temporal aspect of the network.
  arXiv  Detail & Related papers  (2023-08-01T18:20:05Z)
• Temporal Aggregation and Propagation Graph Neural Networks for Dynamic Representation [67.26422477327179]
  Temporal graphs exhibit dynamic interactions between nodes over continuous time. We propose a novel method of temporal graph convolution with the whole neighborhood. Our proposed TAP-GNN outperforms existing temporal graph methods by a large margin in terms of both predictive performance and online inference latency.
  arXiv  Detail & Related papers  (2023-04-15T08:17:18Z)
• EGRC-Net: Embedding-induced Graph Refinement Clustering Network [66.44293190793294]
  We propose a novel graph clustering network called Embedding-Induced Graph Refinement Clustering Network (EGRC-Net) EGRC-Net effectively utilizes the learned embedding to adaptively refine the initial graph and enhance the clustering performance. Our proposed methods consistently outperform several state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-11-19T09:08:43Z)
• Robust Knowledge Adaptation for Dynamic Graph Neural Networks [61.8505228728726]
  We propose Ada-DyGNN: a robust knowledge Adaptation framework via reinforcement learning for Dynamic Graph Neural Networks. Our approach constitutes the first attempt to explore robust knowledge adaptation via reinforcement learning. Experiments on three benchmark datasets demonstrate that Ada-DyGNN achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-07-22T02:06:53Z)
• DyGCN: Dynamic Graph Embedding with Graph Convolutional Network [25.02329024926518]
  We propose an efficient dynamic graph embedding approach, Dynamic Graph Convolutional Network (DyGCN) Our model can update the node embeddings in a time-saving and performance-preserving way.
  arXiv  Detail & Related papers  (2021-04-07T07:28:44Z)
• DynACPD Embedding Algorithm for Prediction Tasks in Dynamic Networks [6.5361928329696335]
  We present novel embedding methods for a dynamic network based on higher order tensor decompositions for tensorial representations of the dynamic network. We demonstrate the power and efficiency of our approach by comparing our algorithms' performance on the link prediction task against an array of current baseline methods.
  arXiv  Detail & Related papers  (2021-03-12T04:36:42Z)
• Dynamic Graph: Learning Instance-aware Connectivity for Neural Networks [78.65792427542672]
  Dynamic Graph Network (DG-Net) is a complete directed acyclic graph, where the nodes represent convolutional blocks and the edges represent connection paths. Instead of using the same path of the network, DG-Net aggregates features dynamically in each node, which allows the network to have more representation ability.
  arXiv  Detail & Related papers  (2020-10-02T16:50:26Z)
• GloDyNE: Global Topology Preserving Dynamic Network Embedding [31.269883917366478]
  Dynamic Network Embedding (DNE) aims to update node embeddings while preserving network topology at each time step. We propose a novel strategy to diversely select the representative nodes over a network, which is coordinated with a new incremental learning paradigm. Experiments show GloDyNE, with a small fraction of nodes being selected, can already achieve the superior or comparable performance.
  arXiv  Detail & Related papers  (2020-08-05T05:10:15Z)
• Policy-GNN: Aggregation Optimization for Graph Neural Networks [60.50932472042379]
  Graph neural networks (GNNs) aim to model the local graph structures and capture the hierarchical patterns by aggregating the information from neighbors. It is a challenging task to develop an effective aggregation strategy for each node, given complex graphs and sparse features. We propose Policy-GNN, a meta-policy framework that models the sampling procedure and message passing of GNNs into a combined learning process.
  arXiv  Detail & Related papers  (2020-06-26T17:03:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.