Dynamic Spiking Framework for Graph Neural Networks

• URL: http://arxiv.org/abs/2401.05373v3
• Date: Tue, 30 Jul 2024 09:05:05 GMT
• Title: Dynamic Spiking Framework for Graph Neural Networks
• Authors: Nan Yin, Mengzhu Wang, Zhenghan Chen, Giulia De Masi, Bin Gu, Huan Xiong,
• Abstract summary: We present a framework named underlineDynamic underlineSpunderlineiking underlineGraph underlineNeural Networks (method) to mitigate the information loss problem. Experiments on three large-scale real-world dynamic graph validate the effectiveness of method.
• Score: 26.08442716817432
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The integration of Spiking Neural Networks (SNNs) and Graph Neural Networks (GNNs) is gradually attracting attention due to the low power consumption and high efficiency in processing the non-Euclidean data represented by graphs. However, as a common problem, dynamic graph representation learning faces challenges such as high complexity and large memory overheads. Current work often uses SNNs instead of Recurrent Neural Networks (RNNs) by using binary features instead of continuous ones for efficient training, which would overlooks graph structure information and leads to the loss of details during propagation. Additionally, optimizing dynamic spiking models typically requires propagation of information across time steps, which increases memory requirements. To address these challenges, we present a framework named \underline{Dy}namic \underline{S}p\underline{i}king \underline{G}raph \underline{N}eural Networks (\method{}). To mitigate the information loss problem, \method{} propagates early-layer information directly to the last layer for information compensation. To accommodate the memory requirements, we apply the implicit differentiation on the equilibrium state, which does not rely on the exact reverse of the forward computation. While traditional implicit differentiation methods are usually used for static situations, \method{} extends it to the dynamic graph setting. Extensive experiments on three large-scale real-world dynamic graph datasets validate the effectiveness of \method{} on dynamic node classification tasks with lower computational costs.

Related papers

• Efficient and Effective Implicit Dynamic Graph Neural Network [42.49148111696576]
  We present Implicit Dynamic Graph Neural Network (IDGNN) a novel implicit neural network for dynamic graphs. A key characteristic of IDGNN is that it demonstrably is well-posed, i.e., it is theoretically guaranteed to have a fixed-point representation.
  arXiv  Detail & Related papers  (2024-06-25T19:07:21Z)
• Gradient Transformation: Towards Efficient and Model-Agnostic Unlearning for Dynamic Graph Neural Networks [66.70786325911124]
  Graph unlearning has emerged as an essential tool for safeguarding user privacy and mitigating the negative impacts of undesirable data. With the increasing prevalence of DGNNs, it becomes imperative to investigate the implementation of dynamic graph unlearning. We propose an effective, efficient, model-agnostic, and post-processing method to implement DGNN unlearning.
  arXiv  Detail & Related papers  (2024-05-23T10:26:18Z)
• Dynamic Graph Message Passing Networks for Visual Recognition [112.49513303433606]
  Modelling long-range dependencies is critical for scene understanding tasks in computer vision. A fully-connected graph is beneficial for such modelling, but its computational overhead is prohibitive. We propose a dynamic graph message passing network, that significantly reduces the computational complexity.
  arXiv  Detail & Related papers  (2022-09-20T14:41:37Z)
• Scaling Up Dynamic Graph Representation Learning via Spiking Neural Networks [23.01100055999135]
  We present a scalable framework, namely SpikeNet, to efficiently capture the temporal and structural patterns of temporal graphs. As a low-power alternative to RNNs, SNNs explicitly model graph dynamics as spike trains of neuron populations. SpikeNet generalizes to a large temporal graph with significantly fewer parameters and computation overheads.
  arXiv  Detail & Related papers  (2022-08-15T09:22:15Z)
• Instant Graph Neural Networks for Dynamic Graphs [18.916632816065935]
  We propose Instant Graph Neural Network (InstantGNN), an incremental approach for the graph representation matrix of dynamic graphs. Our method avoids time-consuming, repetitive computations and allows instant updates on the representation and instant predictions. Our model achieves state-of-the-art accuracy while having orders-of-magnitude higher efficiency than existing methods.
  arXiv  Detail & Related papers  (2022-06-03T03:27:42Z)
• Learning Graph Structure from Convolutional Mixtures [119.45320143101381]
  We propose a graph convolutional relationship between the observed and latent graphs, and formulate the graph learning task as a network inverse (deconvolution) problem. In lieu of eigendecomposition-based spectral methods, we unroll and truncate proximal gradient iterations to arrive at a parameterized neural network architecture that we call a Graph Deconvolution Network (GDN) GDNs can learn a distribution of graphs in a supervised fashion, perform link prediction or edge-weight regression tasks by adapting the loss function, and they are inherently inductive.
  arXiv  Detail & Related papers  (2022-05-19T14:08:15Z)
• DOTIN: Dropping Task-Irrelevant Nodes for GNNs [119.17997089267124]
  Recent graph learning approaches have introduced the pooling strategy to reduce the size of graphs for learning. We design a new approach called DOTIN (underlineDrunderlineopping underlineTask-underlineIrrelevant underlineNodes) to reduce the size of graphs. Our method speeds up GAT by about 50% on graph-level tasks including graph classification and graph edit distance.
  arXiv  Detail & Related papers  (2022-04-28T12:00:39Z)
• Neural Graph Matching for Pre-training Graph Neural Networks [72.32801428070749]
  Graph neural networks (GNNs) have been shown powerful capacity at modeling structural data. We present a novel Graph Matching based GNN Pre-Training framework, called GMPT. The proposed method can be applied to fully self-supervised pre-training and coarse-grained supervised pre-training.
  arXiv  Detail & Related papers  (2022-03-03T09:53:53Z)
• Efficient-Dyn: Dynamic Graph Representation Learning via Event-based Temporal Sparse Attention Network [2.0047096160313456]
  Dynamic graph neural networks have received more and more attention from researchers. We propose a novel dynamic graph neural network, Efficient-Dyn. It adaptively encodes temporal information into a sequence of patches with an equal amount of temporal-topological structure.
  arXiv  Detail & Related papers  (2022-01-04T23:52:24Z)
• Learning to Evolve on Dynamic Graphs [5.1521870302904125]
  Learning to Evolve on Dynamic Graphs (LEDG) is a novel algorithm that jointly learns graph information and time information. LEDG is model-agnostic and can train any message passing based graph neural network (GNN) on dynamic graphs.
  arXiv  Detail & Related papers  (2021-11-13T04:09:30Z)
• Binary Graph Neural Networks [69.51765073772226]
  Graph Neural Networks (GNNs) have emerged as a powerful and flexible framework for representation learning on irregular data. In this paper, we present and evaluate different strategies for the binarization of graph neural networks. We show that through careful design of the models, and control of the training process, binary graph neural networks can be trained at only a moderate cost in accuracy on challenging benchmarks.
  arXiv  Detail & Related papers  (2020-12-31T18:48:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.