Dynamic Graph Transformer with Correlated Spatial-Temporal Positional Encoding

• URL: http://arxiv.org/abs/2407.16959v1
• Date: Wed, 24 Jul 2024 02:56:22 GMT
• Title: Dynamic Graph Transformer with Correlated Spatial-Temporal Positional Encoding
• Abstract summary: A fundamental requirement for representation learning in Continuous-Time Dynamic Graphs (CTDGs) is the appropriate estimation and preservation of proximity. We propose a novel Correlated Spatial-Temporal Positional representation (CorDGT) that efficiently retains the evolving spatial-temporal proximity for effective node learning in CTDGs.
• Score: 29.81572367536339
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning effective representations for Continuous-Time Dynamic Graphs (CTDGs) has garnered significant research interest, largely due to its powerful capabilities in modeling complex interactions between nodes. A fundamental and crucial requirement for representation learning in CTDGs is the appropriate estimation and preservation of proximity. However, due to the sparse and evolving characteristics of CTDGs, the spatial-temporal properties inherent in high-order proximity remain largely unexplored. Despite its importance, this property presents significant challenges due to the computationally intensive nature of personalized interaction intensity estimation and the dynamic attributes of CTDGs. To this end, we propose a novel Correlated Spatial-Temporal Positional encoding that incorporates a parameter-free personalized interaction intensity estimation under the weak assumption of the Poisson Point Process. Building on this, we introduce the Dynamic Graph Transformer with \Correlated Spatial-Temporal Positional Encoding (CorDGT), which efficiently retains the evolving spatial-temporal high-order proximity for effective node representation learning in CTDGs. Extensive experiments on seven small and two large-scale datasets demonstrate the superior performance and scalability of the proposed CorDGT.

Related papers

• DyG$^2$T: Modeling Object Dynamics with 3D Gaussian Temporal-Spatial Particle Graph Transformer [86.4618122245946]
  DyG$2$T is a dynamics modeling framework that infers object motion trajectories by spatially completing and temporally discriminating Key Point representations.<n>Experiments on both synthetic and real-world datasets demonstrate that DyG$2$T achieves accurate dynamics modeling and reasoning.
  arXiv  Detail & Related papers  (2026-08-19T03:44:28Z)
• Electromyography-Based Gesture Recognition: Hierarchical Feature Extraction for Enhanced Spatial-Temporal Dynamics [0.7083699704958353]
  We propose a lightweight squeeze-excitation deep learning-based multi stream spatial temporal dynamics time-varying feature extraction approach. The proposed model was tested on the Ninapro DB2, DB4, and DB5 datasets, achieving accuracy rates of 96.41%, 92.40%, and 93.34%, respectively.
  arXiv  Detail & Related papers  (2025-04-04T07:11:12Z)
• Dynamic Graph Structure Estimation for Learning Multivariate Point Process using Spiking Neural Networks [14.77536193242342]
  Spiking Dynamic Graph Network is a novel framework that leverages the temporal processing capabilities of spiking neural networks (SNNs) and spike-dependent plasticity (STD-P) It adapts to any dataset by learning dynamic-temporal dependencies directly from event data, enhancing generalizability and modeling. Our evaluations conducted on both synthetic and real-world datasets including NYC Taxi, 911 Reddit, and Stack Overflow, demonstrate superior accuracy while maintaining computational efficiency.
  arXiv  Detail & Related papers  (2025-04-01T23:23:10Z)
• Conservation-informed Graph Learning for Spatiotemporal Dynamics Prediction [84.26340606752763]
  In this paper, we introduce the conservation-informed GNN (CiGNN), an end-to-end explainable learning framework. The network is designed to conform to the general symmetry conservation law via symmetry where conservative and non-conservative information passes over a multiscale space by a latent temporal marching strategy. Results demonstrate that CiGNN exhibits remarkable baseline accuracy and generalizability, and is readily applicable to learning for prediction of varioustemporal dynamics.
  arXiv  Detail & Related papers  (2024-12-30T13:55:59Z)
• STDHL: Spatio-Temporal Dynamic Hypergraph Learning for Wind Power Forecasting [5.003934238878358]
  We present a dynamic hypergraph learning (STDHL) model to represent spatial features among wind farms. STDHL model incorporates novel dynamic hypergraph convolutional layer to model dynamic spatial correlations and grouped temporal convolutional layer for channel-independent temporal modeling.
  arXiv  Detail & Related papers  (2024-12-16T02:43:29Z)
• A Cosmic-Scale Benchmark for Symmetry-Preserving Data Processing [1.96862953848735]
  We benchmark the ability of graph neural networks to simultaneously capture local clustering environments and long-range correlations. We find that current architectures fail to capture information from long-range correlations as effectively as domain-specific baselines.
  arXiv  Detail & Related papers  (2024-10-27T16:58:48Z)
• Degree-Conscious Spiking Graph for Cross-Domain Adaptation [51.58506501415558]
  Spiking Graph Networks (SGNs) have demonstrated significant potential in graph classification.<n>We introduce a novel framework named Degree-Consicious Spiking Graph for Cross-Domain Adaptation (DeSGraDA)<n>DeSGraDA enhances generalization across domains with three key components.
  arXiv  Detail & Related papers  (2024-10-09T13:45:54Z)
• Efficient High-Resolution Visual Representation Learning with State Space Model for Human Pose Estimation [60.80423207808076]
  Capturing long-range dependencies while preserving high-resolution visual representations is crucial for dense prediction tasks such as human pose estimation.<n>We propose the Dynamic Visual State Space (DVSS) block, which augments visual state space models with multi-scale convolutional operations.<n>We build HRVMamba, a novel model for efficient high-resolution representation learning.
  arXiv  Detail & Related papers  (2024-10-04T06:19:29Z)
• DyG-Mamba: Continuous State Space Modeling on Dynamic Graphs [59.434893231950205]
  Dynamic graph learning aims to uncover evolutionary laws in real-world systems. We propose DyG-Mamba, a new continuous state space model for dynamic graph learning. We show that DyG-Mamba achieves state-of-the-art performance on most datasets.
  arXiv  Detail & Related papers  (2024-08-13T15:21:46Z)
• A Novel Spatiotemporal Coupling Graph Convolutional Network [0.18130068086063336]
  This paper presents a novel Graph Contemporalal Networks (GCNs)-based dynamic estimator namely Spatio Coupling GCN (SCG) model with the three-fold ideas as below. The results demonstrate that SCG realizes higher accuracy compared with the state-of-the-arts, illustrating it can learn powerful representations to users and cloud services.
  arXiv  Detail & Related papers  (2024-08-09T02:02:01Z)
• Graph Convolutional Network with Connectivity Uncertainty for EEG-based Emotion Recognition [20.655367200006076]
  This study introduces the distribution-based uncertainty method to represent spatial dependencies and temporal-spectral relativeness in EEG signals. The graph mixup technique is employed to enhance latent connected edges and mitigate noisy label issues. We evaluate our approach on two widely used datasets, namely SEED and SEEDIV, for emotion recognition tasks.
  arXiv  Detail & Related papers  (2023-10-22T03:47:11Z)
• Label Deconvolution for Node Representation Learning on Large-scale Attributed Graphs against Learning Bias [75.44877675117749]
  We propose an efficient label regularization technique, namely Label Deconvolution (LD), to alleviate the learning bias by a novel and highly scalable approximation to the inverse mapping of GNNs. Experiments demonstrate LD significantly outperforms state-of-the-art methods on Open Graph datasets Benchmark.
  arXiv  Detail & Related papers  (2023-09-26T13:09:43Z)
• Learning in latent spaces improves the predictive accuracy of deep neural operators [0.0]
  L-DeepONet is an extension of standard DeepONet, which leverages latent representations of high-dimensional PDE input and output functions identified with suitable autoencoders. We show that L-DeepONet outperforms the standard approach in terms of both accuracy and computational efficiency across diverse time-dependent PDEs.
  arXiv  Detail & Related papers  (2023-04-15T17:13:09Z)
• EasyDGL: Encode, Train and Interpret for Continuous-time Dynamic Graph Learning [92.71579608528907]
  This paper aims to design an easy-to-use pipeline (termed as EasyDGL) composed of three key modules with both strong ability fitting and interpretability. EasyDGL can effectively quantify the predictive power of frequency content that a model learn from the evolving graph data.
  arXiv  Detail & Related papers  (2023-03-22T06:35:08Z)
• DepthFormer: Exploiting Long-Range Correlation and Local Information for Accurate Monocular Depth Estimation [50.08080424613603]
  Long-range correlation is essential for accurate monocular depth estimation. We propose to leverage the Transformer to model this global context with an effective attention mechanism. Our proposed model, termed DepthFormer, surpasses state-of-the-art monocular depth estimation methods with prominent margins.
  arXiv  Detail & Related papers  (2022-03-27T05:03:56Z)
• Cyclic Graph Attentive Match Encoder (CGAME): A Novel Neural Network For OD Estimation [8.398623478484248]
  Origin-Destination Estimation plays an important role in traffic management and traffic simulation in the era of Intelligent Transportation System (ITS) Previous model-based models face the under-determined challenge, thus desperate demand for additional assumptions and extra data exists. We propose Cyclic Graph Attentive Matching (C-GAME) based on a novel Graph Matcher with double-layer attention mechanism.
  arXiv  Detail & Related papers  (2021-11-26T08:57:21Z)
• Spatial-Temporal Correlation and Topology Learning for Person Re-Identification in Videos [78.45050529204701]
  We propose a novel framework to pursue discriminative and robust representation by modeling cross-scale spatial-temporal correlation. CTL utilizes a CNN backbone and a key-points estimator to extract semantic local features from human body. It explores a context-reinforced topology to construct multi-scale graphs by considering both global contextual information and physical connections of human body.
  arXiv  Detail & Related papers  (2021-04-15T14:32:12Z)
• On the spatial attention in Spatio-Temporal Graph Convolutional Networks for skeleton-based human action recognition [97.14064057840089]
  Graphal networks (GCNs) promising performance in skeleton-based human action recognition by modeling a sequence of skeletons as a graph. Most of the recently proposed G-temporal-based methods improve the performance by learning the graph structure at each layer of the network.
  arXiv  Detail & Related papers  (2020-11-07T19:03:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.