Beyond Attention: Learning Spatio-Temporal Dynamics with Emergent Interpretable Topologies

• URL: http://arxiv.org/abs/2506.00770v1
• Date: Sun, 01 Jun 2025 01:27:32 GMT
• Title: Beyond Attention: Learning Spatio-Temporal Dynamics with Emergent Interpretable Topologies
• Authors: Sai Vamsi Alisetti, Vikas Kalagi, Sanjukta Krishnagopal,
• Abstract summary: Graph Attention Networks (GATs) are popular for modeling spatial dependencies, but they rely on predefined adjacency structures and dynamic attention scores.<n>We propose InterGAT, a simplified alternative that replaces masked attention with a fully learnable, symmetric node interaction matrix.<n>Our framework, InterGAT-GRU, outperforms the baseline GAT-GRU in forecasting accuracy, achieving at least a 21% improvement on the SZ-Taxi dataset and a 6% improvement on the Los-Loop dataset.
• Score: 0.20482269513546453
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Spatio-temporal forecasting is critical in applications such as traffic prediction, energy demand modeling, and weather monitoring. While Graph Attention Networks (GATs) are popular for modeling spatial dependencies, they rely on predefined adjacency structures and dynamic attention scores, introducing inductive biases and computational overhead that can obscure interpretability. We propose InterGAT, a simplified alternative to GAT that replaces masked attention with a fully learnable, symmetric node interaction matrix, capturing latent spatial relationships without relying on fixed graph topologies. Our framework, InterGAT-GRU, which incorporates a GRU-based temporal decoder, outperforms the baseline GAT-GRU in forecasting accuracy, achieving at least a 21% improvement on the SZ-Taxi dataset and a 6% improvement on the Los-Loop dataset across all forecasting horizons (15 to 60 minutes). Additionally, we observed reduction in training time by 60-70% compared to GAT-GRU baseline. Crucially, the learned interaction matrix reveals interpretable structure: it recovers sparse, topology-aware attention patterns that align with community structure. Spectral and clustering analyses show that the model captures both localized and global dynamics, offering insights into the functional topology driving predictions. This highlights how structure learning can simultaneously support prediction, computational efficiency, and topological interpretabil-ity in dynamic graph-based domains.

Related papers

• Topology-Aware Conformal Prediction for Stream Networks [54.505880918607296]
  We propose Spatio-Temporal Adaptive Conformal Inference (textttCISTA), a novel framework that integrates network topology and temporal dynamics into the conformal prediction framework.<n>Our results show that textttCISTA effectively balances prediction efficiency and coverage, outperforming existing conformal prediction methods for stream networks.
  arXiv  Detail & Related papers  (2025-03-06T21:21:15Z)
• Utilizing Graph Neural Networks for Effective Link Prediction in Microservice Architectures [0.0]
  This study introduces a Graph Neural Network based approach, specifically using a Graph Attention Network GAT, for link prediction in microservice Call Graphs.<n>Unlike social networks, where interactions tend to occur sporadically and are often less frequent, microservice Call Graphs involve highly frequent and time sensitive interactions.<n>Using real world data, we evaluate our model across performance metrics such as AUC, Precision, Recall, and F1 Score, demonstrating its high accuracy and robustness in predicting microservice interactions.
  arXiv  Detail & Related papers  (2025-01-25T01:51:30Z)
• Graph Pruning Based Spatial and Temporal Graph Convolutional Network with Transfer Learning for Traffic Prediction [0.0]
  This study proposes a novel Spatial-temporal Convolutional Network (TL-GPSTGN) based on graph pruning and transfer learning framework.<n>The results demonstrate the exceptional predictive accuracy of TL-GPSTGN on a single dataset, as well as its robust migration performance across different datasets.
  arXiv  Detail & Related papers  (2024-09-25T00:59:23Z)
• Context-aware knowledge graph framework for traffic speed forecasting using graph neural network [5.96412390867876]
  This study proposes a novel context-aware knowledge graph (CKG) framework to enhance traffic speed forecasting.<n>A CKG-GNN model, combining the CKG, dual-view multi-head self-attention (MHSA), and graph neural network (GNN), is then designed to predict traffic speed.
  arXiv  Detail & Related papers  (2024-07-25T01:52:12Z)
• SAMSGL: Series-Aligned Multi-Scale Graph Learning for Spatio-Temporal Forecasting [9.013416216828361]
  We present a Series-Aligned Multi-Scale Graph Learning (SGL) framework, aiming to enhance forecasting performance. In this work, we propose a series-aligned graph layer to facilitate the aggregation of non-delayed graph signals. We conduct experiments on meteorological and traffic forecasting datasets, which demonstrate its effectiveness and superiority.
  arXiv  Detail & Related papers  (2023-12-05T10:37:54Z)
• FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective [48.00240550685946]
  Current state-of-the-art graph neural network (GNN)-based forecasting methods usually require both graph networks (e.g., GCN) and temporal networks (e.g., LSTM) to capture inter-series (spatial) dynamics and intra-series (temporal) dependencies, respectively. We propose a novel Fourier Graph Neural Network (FourierGNN) by stacking our proposed Fourier Graph Operator (FGO) to perform matrix multiplications in Fourier space. Our experiments on seven datasets have demonstrated superior performance with higher efficiency and fewer parameters compared with state-of-the-
  arXiv  Detail & Related papers  (2023-11-10T17:13:26Z)
• Deep Temporal Graph Clustering [77.02070768950145]
  We propose a general framework for deep Temporal Graph Clustering (GC) GC introduces deep clustering techniques to suit the interaction sequence-based batch-processing pattern of temporal graphs. Our framework can effectively improve the performance of existing temporal graph learning methods.
  arXiv  Detail & Related papers  (2023-05-18T06:17:50Z)
• EasyDGL: Encode, Train and Interpret for Continuous-time Dynamic Graph Learning [92.71579608528907]
  This paper aims to design an easy-to-use pipeline (termed as EasyDGL) composed of three key modules with both strong ability fitting and interpretability. EasyDGL can effectively quantify the predictive power of frequency content that a model learn from the evolving graph data.
  arXiv  Detail & Related papers  (2023-03-22T06:35:08Z)
• Optimal Propagation for Graph Neural Networks [51.08426265813481]
  We propose a bi-level optimization approach for learning the optimal graph structure. We also explore a low-rank approximation model for further reducing the time complexity.
  arXiv  Detail & Related papers  (2022-05-06T03:37:00Z)
• Spatio-Temporal Latent Graph Structure Learning for Traffic Forecasting [6.428566223253948]
  We propose a new traffic forecasting framework--S-Temporal Latent Graph Structure Learning networks (ST-LGSL) The model employs a graph based on Multilayer perceptron and K-Nearest Neighbor, which learns the latent graph topological information from the entire data. With the dependencies-kNN based on ground-truth adjacency matrix and similarity metric in kNN, ST-LGSL aggregates the top focusing on geography and node similarity.
  arXiv  Detail & Related papers  (2022-02-25T10:02:49Z)
• PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning [109.84770951839289]
  We present PredRNN, a new recurrent network for learning visual dynamics from historical context. We show that our approach obtains highly competitive results on three standard datasets.
  arXiv  Detail & Related papers  (2021-03-17T08:28:30Z)
• On the spatial attention in Spatio-Temporal Graph Convolutional Networks for skeleton-based human action recognition [97.14064057840089]
  Graphal networks (GCNs) promising performance in skeleton-based human action recognition by modeling a sequence of skeletons as a graph. Most of the recently proposed G-temporal-based methods improve the performance by learning the graph structure at each layer of the network.
  arXiv  Detail & Related papers  (2020-11-07T19:03:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.