DyG-Mamba: Continuous State Space Modeling on Dynamic Graphs

• URL: http://arxiv.org/abs/2408.06966v1
• Date: Tue, 13 Aug 2024 15:21:46 GMT
• Title: DyG-Mamba: Continuous State Space Modeling on Dynamic Graphs
• Authors: Dongyuan Li, Shiyin Tan, Ying Zhang, Ming Jin, Shirui Pan, Manabu Okumura, Renhe Jiang,
• Abstract summary: Dynamic graph learning aims to uncover evolutionary laws in real-world systems. We propose DyG-Mamba, a new continuous state space model for dynamic graph learning. We show that DyG-Mamba achieves state-of-the-art performance on most datasets.
• Score: 59.434893231950205
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dynamic graph learning aims to uncover evolutionary laws in real-world systems, enabling accurate social recommendation (link prediction) or early detection of cancer cells (classification). Inspired by the success of state space models, e.g., Mamba, for efficiently capturing long-term dependencies in language modeling, we propose DyG-Mamba, a new continuous state space model (SSM) for dynamic graph learning. Specifically, we first found that using inputs as control signals for SSM is not suitable for continuous-time dynamic network data with irregular sampling intervals, resulting in models being insensitive to time information and lacking generalization properties. Drawing inspiration from the Ebbinghaus forgetting curve, which suggests that memory of past events is strongly correlated with time intervals rather than specific details of the events themselves, we directly utilize irregular time spans as control signals for SSM to achieve significant robustness and generalization. Through exhaustive experiments on 12 datasets for dynamic link prediction and dynamic node classification tasks, we found that DyG-Mamba achieves state-of-the-art performance on most of the datasets, while also demonstrating significantly improved computation and memory efficiency.

Related papers

• ScaDyG:A New Paradigm for Large-scale Dynamic Graph Learning [31.629956388962814]
  ScaDyG is a time-aware scalable learning paradigm for dynamic graph networks. experiments on 12 datasets demonstrate that ScaDyG performs comparably well or even outperforms other SOTA methods in both node and link-level downstream tasks.
  arXiv  Detail & Related papers  (2025-01-27T12:39:16Z)
• DG-Mamba: Robust and Efficient Dynamic Graph Structure Learning with Selective State Space Models [16.435352947791923]
  We propose a novel Dynamic Graph structure learning framework with the Selective State Space Models (Mamba) Our framework is superior to state-of-the-art baselines against adversarial attacks.
  arXiv  Detail & Related papers  (2024-12-11T07:32:38Z)
• LLM-Based Multi-Agent Systems are Scalable Graph Generative Models [73.28294528654885]
  GraphAgent-Generator (GAG) is a novel simulation-based framework for dynamic, text-attributed social graph generation. GAG simulates the temporal node and edge generation processes for zero-shot social graph generation. The resulting graphs exhibit adherence to seven key macroscopic network properties, achieving an 11% improvement in microscopic graph structure metrics.
  arXiv  Detail & Related papers  (2024-10-13T12:57:08Z)
• DyGMamba: Efficiently Modeling Long-Term Temporal Dependency on Continuous-Time Dynamic Graphs with State Space Models [26.989676396289145]
  We present DyGMamba, a learning model for continuous-time dynamic graphs (CTDGs) We show that DyGMamba achieves state-of-the-art in most cases.
  arXiv  Detail & Related papers  (2024-08-08T18:25:14Z)
• State Space Models on Temporal Graphs: A First-Principles Study [30.531930200222423]
  Research on deep graph learning has shifted from static graphs to temporal graphs in response to real-world complex systems that exhibit dynamic behaviors. Sequence models such as RNNs or Transformers have long been the predominant backbone networks for modeling such temporal graphs. We develop GraphSSM, a graph state space model for modeling the dynamics of temporal graphs.
  arXiv  Detail & Related papers  (2024-06-03T02:56:11Z)
• Input Snapshots Fusion for Scalable Discrete-Time Dynamic Graph Neural Networks [27.616083395612595]
  We propose SFDyG, which combines Hawkes processes with graph neural networks to capture temporal and structural patterns in dynamic graphs effectively. By fusing multiple snapshots into a single temporal graph, SFDyG decouples computational complexity from the number of snapshots, enabling efficient full-batch and mini-batch training.
  arXiv  Detail & Related papers  (2024-05-11T10:05:55Z)
• TimeGraphs: Graph-based Temporal Reasoning [64.18083371645956]
  TimeGraphs is a novel approach that characterizes dynamic interactions as a hierarchical temporal graph. Our approach models the interactions using a compact graph-based representation, enabling adaptive reasoning across diverse time scales. We evaluate TimeGraphs on multiple datasets with complex, dynamic agent interactions, including a football simulator, the Resistance game, and the MOMA human activity dataset.
  arXiv  Detail & Related papers  (2024-01-06T06:26:49Z)
• EasyDGL: Encode, Train and Interpret for Continuous-time Dynamic Graph Learning [92.71579608528907]
  This paper aims to design an easy-to-use pipeline (termed as EasyDGL) composed of three key modules with both strong ability fitting and interpretability. EasyDGL can effectively quantify the predictive power of frequency content that a model learn from the evolving graph data.
  arXiv  Detail & Related papers  (2023-03-22T06:35:08Z)
• Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs [65.18780403244178]
  We propose a continuous model to forecast Multivariate Time series with dynamic Graph neural Ordinary Differential Equations (MTGODE) Specifically, we first abstract multivariate time series into dynamic graphs with time-evolving node features and unknown graph structures. Then, we design and solve a neural ODE to complement missing graph topologies and unify both spatial and temporal message passing.
  arXiv  Detail & Related papers  (2022-02-17T02:17:31Z)
• Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition [79.33539539956186]
  We propose a simple method to disentangle multi-scale graph convolutions and a unified spatial-temporal graph convolutional operator named G3D. By coupling these proposals, we develop a powerful feature extractor named MS-G3D based on which our model outperforms previous state-of-the-art methods on three large-scale datasets.
  arXiv  Detail & Related papers  (2020-03-31T11:28:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.