DyG-Mamba: Continuous State Space Modeling on Dynamic Graphs

• URL: http://arxiv.org/abs/2408.06966v1
• Date: Tue, 13 Aug 2024 15:21:46 GMT
• Title: DyG-Mamba: Continuous State Space Modeling on Dynamic Graphs
• Authors: Dongyuan Li, Shiyin Tan, Ying Zhang, Ming Jin, Shirui Pan, Manabu Okumura, Renhe Jiang,
• Abstract summary: Dynamic graph learning aims to uncover evolutionary laws in real-world systems. We propose DyG-Mamba, a new continuous state space model for dynamic graph learning. We show that DyG-Mamba achieves state-of-the-art performance on most datasets.
• Score: 59.434893231950205
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dynamic graph learning aims to uncover evolutionary laws in real-world systems, enabling accurate social recommendation (link prediction) or early detection of cancer cells (classification). Inspired by the success of state space models, e.g., Mamba, for efficiently capturing long-term dependencies in language modeling, we propose DyG-Mamba, a new continuous state space model (SSM) for dynamic graph learning. Specifically, we first found that using inputs as control signals for SSM is not suitable for continuous-time dynamic network data with irregular sampling intervals, resulting in models being insensitive to time information and lacking generalization properties. Drawing inspiration from the Ebbinghaus forgetting curve, which suggests that memory of past events is strongly correlated with time intervals rather than specific details of the events themselves, we directly utilize irregular time spans as control signals for SSM to achieve significant robustness and generalization. Through exhaustive experiments on 12 datasets for dynamic link prediction and dynamic node classification tasks, we found that DyG-Mamba achieves state-of-the-art performance on most of the datasets, while also demonstrating significantly improved computation and memory efficiency.

Related papers

• DyGMamba: Efficiently Modeling Long-Term Temporal Dependency on Continuous-Time Dynamic Graphs with State Space Models [26.989676396289145]
  We present DyGMamba, a learning model for continuous-time dynamic graphs (CTDGs) We show that DyGMamba achieves state-of-the-art in most cases.
  arXiv  Detail & Related papers  (2024-08-08T18:25:14Z)
• Long Range Propagation on Continuous-Time Dynamic Graphs [18.5534584418248]
  Continuous-Time Graph Anti-Symmetric Network (CTAN) is designed for efficient propagation of information. We show how CTAN's empirical performance on synthetic long-range benchmarks and real-world benchmarks is superior to other methods.
  arXiv  Detail & Related papers  (2024-06-04T19:42:19Z)
• State Space Models on Temporal Graphs: A First-Principles Study [30.531930200222423]
  Research on deep graph learning has shifted from static graphs to temporal graphs in response to real-world complex systems that exhibit dynamic behaviors. Sequence models such as RNNs or Transformers have long been the predominant backbone networks for modeling such temporal graphs. We develop GraphSSM, a graph state space model for modeling the dynamics of temporal graphs.
  arXiv  Detail & Related papers  (2024-06-03T02:56:11Z)
• A Poisson-Gamma Dynamic Factor Model with Time-Varying Transition Dynamics [51.147876395589925]
  A non-stationary PGDS is proposed to allow the underlying transition matrices to evolve over time. A fully-conjugate and efficient Gibbs sampler is developed to perform posterior simulation. Experiments show that, in comparison with related models, the proposed non-stationary PGDS achieves improved predictive performance.
  arXiv  Detail & Related papers  (2024-02-26T04:39:01Z)
• TimeGraphs: Graph-based Temporal Reasoning [64.18083371645956]
  TimeGraphs is a novel approach that characterizes dynamic interactions as a hierarchical temporal graph. Our approach models the interactions using a compact graph-based representation, enabling adaptive reasoning across diverse time scales. We evaluate TimeGraphs on multiple datasets with complex, dynamic agent interactions, including a football simulator, the Resistance game, and the MOMA human activity dataset.
  arXiv  Detail & Related papers  (2024-01-06T06:26:49Z)
• Backbone-based Dynamic Graph Spatio-Temporal Network for Epidemic Forecasting [3.382729969842304]
  Accurate epidemic forecasting is a critical task in controlling disease transmission. Many deep learning-based models focus only on static or dynamic graphs when constructing spatial information. We propose a novel model called Backbone-based Dynamic Graph Spatio-Temporal Network (BDGSTN)
  arXiv  Detail & Related papers  (2023-12-01T10:34:03Z)
• Exploring Time Granularity on Temporal Graphs for Dynamic Link Prediction in Real-world Networks [0.48346848229502226]
  Dynamic Graph Neural Networks (DGNNs) have emerged as the predominant approach for processing dynamic graph-structured data. In this paper, we explore the impact of time granularity when training DGNNs on dynamic graphs through extensive experiments.
  arXiv  Detail & Related papers  (2023-11-21T00:34:53Z)
• EasyDGL: Encode, Train and Interpret for Continuous-time Dynamic Graph Learning [92.71579608528907]
  This paper aims to design an easy-to-use pipeline (termed as EasyDGL) composed of three key modules with both strong ability fitting and interpretability. EasyDGL can effectively quantify the predictive power of frequency content that a model learn from the evolving graph data.
  arXiv  Detail & Related papers  (2023-03-22T06:35:08Z)
• Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs [65.18780403244178]
  We propose a continuous model to forecast Multivariate Time series with dynamic Graph neural Ordinary Differential Equations (MTGODE) Specifically, we first abstract multivariate time series into dynamic graphs with time-evolving node features and unknown graph structures. Then, we design and solve a neural ODE to complement missing graph topologies and unify both spatial and temporal message passing.
  arXiv  Detail & Related papers  (2022-02-17T02:17:31Z)
• Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition [79.33539539956186]
  We propose a simple method to disentangle multi-scale graph convolutions and a unified spatial-temporal graph convolutional operator named G3D. By coupling these proposals, we develop a powerful feature extractor named MS-G3D based on which our model outperforms previous state-of-the-art methods on three large-scale datasets.
  arXiv  Detail & Related papers  (2020-03-31T11:28:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.