MultiSPANS: A Multi-range Spatial-Temporal Transformer Network for Traffic Forecast via Structural Entropy Optimization

• URL: http://arxiv.org/abs/2311.02880v1
• Date: Mon, 6 Nov 2023 05:19:06 GMT
• Title: MultiSPANS: A Multi-range Spatial-Temporal Transformer Network for Traffic Forecast via Structural Entropy Optimization
• Abstract summary: We propose MultiSPANS to model complex multi-range dependencies using localtemporal and road network knowledge. We employ structural entropy theory to optimize the spatial attention mechanism. Experiments demonstrate the superiority of the presented framework over several state-of-the-art methods in real-world traffic datasets.
• Score: 34.27143920444722
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traffic forecasting is a complex multivariate time-series regression task of paramount importance for traffic management and planning. However, existing approaches often struggle to model complex multi-range dependencies using local spatiotemporal features and road network hierarchical knowledge. To address this, we propose MultiSPANS. First, considering that an individual recording point cannot reflect critical spatiotemporal local patterns, we design multi-filter convolution modules for generating informative ST-token embeddings to facilitate attention computation. Then, based on ST-token and spatial-temporal position encoding, we employ the Transformers to capture long-range temporal and spatial dependencies. Furthermore, we introduce structural entropy theory to optimize the spatial attention mechanism. Specifically, The structural entropy minimization algorithm is used to generate optimal road network hierarchies, i.e., encoding trees. Based on this, we propose a relative structural entropy-based position encoding and a multi-head attention masking scheme based on multi-layer encoding trees. Extensive experiments demonstrate the superiority of the presented framework over several state-of-the-art methods in real-world traffic datasets, and the longer historical windows are effectively utilized. The code is available at https://github.com/SELGroup/MultiSPANS.

Related papers

• Eliminating Propagation Delay: Attention-Based Spatial-Temporal Fusion Graph Convolution Network for Traffic Flow Prediction [21.03782087892017]
  We propose a novel network called the Attention-Based Spatial-Temporal Fusion Graph Convolution Network (A-STFGCN)<n>We design a spatial-temporal fusion block to extract the spatial-temporal feature correlations with propagation delay errors removed.<n>Our method achieves the best overall performance while having good computation and data utilization efficiency.
  arXiv  Detail & Related papers  (2026-07-27T10:55:20Z)
• Time-TK: A Multi-Offset Temporal Interaction Framework Combining Transformer and Kolmogorov-Arnold Networks for Time Series Forecasting [6.1337977581640075]
  Existing methods typically employ a strategy of embedding each time step as an independent token.<n>Time-TK significantly outperforms all baseline models, achieving state-of-the-art forecasting accuracy.
  arXiv  Detail & Related papers  (2026-01-30T10:11:51Z)
• Wireless Traffic Prediction with Large Language Model [54.07581399989292]
  TIDES is a novel framework that captures spatial-temporal correlations for wireless traffic prediction.<n> TIDES achieves efficient adaptation to domain-specific patterns without incurring excessive training overhead.<n>Our results indicate that integrating spatial awareness into LLM-based predictors is the key to unlocking scalable and intelligent network management in future 6G systems.
  arXiv  Detail & Related papers  (2025-12-19T04:47:40Z)
• An Enhanced Pyramid Feature Network Based on Long-Range Dependencies for Multi-Organ Medical Image Segmentation [12.785846392087144]
  We propose a new deep-learning network called LamFormer for fine-grained segmentation tasks across multiple organs.<n>LamFormer outperforms existing segmentation methods on seven complex and diverse datasets.
  arXiv  Detail & Related papers  (2025-09-29T06:57:11Z)
• Flow-Through Tensors: A Unified Computational Graph Architecture for Multi-Layer Transportation Network Optimization [20.685856719515026]
  Flow Throughs (FTT) is a unified computational graph architecture that connects origin destination flows, path, probabilities and link travel times as interconnected tensors.<n>Our framework makes three key contributions: first, it establishes a consistent mathematical structure that enables gradient-based optimization across previously separate modeling elements.<n>Second, it supports multidimensional analysis of traffic patterns over time, space, and user groups with precise quantification of system efficiency.
  arXiv  Detail & Related papers  (2025-06-30T06:42:23Z)
• Multivariate Long-term Time Series Forecasting with Fourier Neural Filter [55.09326865401653]
  We introduce FNF as the backbone and DBD as architecture to provide excellent learning capabilities and optimal learning pathways for spatial-temporal modeling.<n>We show that FNF unifies local time-domain and global frequency-domain information processing within a single backbone that extends naturally to spatial modeling.
  arXiv  Detail & Related papers  (2025-06-10T18:40:20Z)
• DP-LET: An Efficient Spatio-Temporal Network Traffic Prediction Framework [13.65226228907662]
  DP-LET is an efficient feature-temporal network traffic prediction framework. DP-LET consists of a data processing module, a local feature enhancement module, and a Transformer-based prediction module. A real-world cellular traffic prediction demonstrates the practicality of DP-LET.
  arXiv  Detail & Related papers  (2025-04-04T02:52:43Z)
• Cross-Domain Transfer Learning using Attention Latent Features for Multi-Agent Trajectory Prediction [4.292918274985369]
  We propose a novel spatial-temporal trajectory prediction framework that performs cross-domain adaption on the attention representation of a Transformer-based model. A graph convolutional network is also integrated to construct dynamic graph feature embeddings that accurately model the complex spatial-temporal interactions between the multi-agent vehicles.
  arXiv  Detail & Related papers  (2024-11-09T06:39:44Z)
• Adaptive Hierarchical SpatioTemporal Network for Traffic Forecasting [70.66710698485745]
  We propose an Adaptive Hierarchical SpatioTemporal Network (AHSTN) to promote traffic forecasting. AHSTN exploits the spatial hierarchy and modeling multi-scale spatial correlations. Experiments on two real-world datasets show that AHSTN achieves better performance over several strong baselines.
  arXiv  Detail & Related papers  (2023-06-15T14:50:27Z)
• FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification [53.55504611255664]
  FormerTime is a hierarchical representation model for improving the classification capacity for the multivariate time series classification task. It exhibits three aspects of merits: (1) learning hierarchical multi-scale representations from time series data, (2) inheriting the strength of both transformers and convolutional networks, and (3) tacking the efficiency challenges incurred by the self-attention mechanism.
  arXiv  Detail & Related papers  (2023-02-20T07:46:14Z)
• Dynamic Spatial Sparsification for Efficient Vision Transformers and Convolutional Neural Networks [88.77951448313486]
  We present a new approach for model acceleration by exploiting spatial sparsity in visual data. We propose a dynamic token sparsification framework to prune redundant tokens. We extend our method to hierarchical models including CNNs and hierarchical vision Transformers.
  arXiv  Detail & Related papers  (2022-07-04T17:00:51Z)
• TMS: A Temporal Multi-scale Backbone Design for Speaker Embedding [60.292702363839716]
  Current SOTA backbone networks for speaker embedding are designed to aggregate multi-scale features from an utterance with multi-branch network architectures for speaker representation. We propose an effective temporal multi-scale (TMS) model where multi-scale branches could be efficiently designed in a speaker embedding network almost without increasing computational costs.
  arXiv  Detail & Related papers  (2022-03-17T05:49:35Z)
• Traffic Flow Forecasting with Spatial-Temporal Graph Diffusion Network [39.65520262751766]
  We develop a new traffic prediction framework-Spatial-Temporal Graph Diffusion Network (ST-GDN) In particular, ST-GDN is a hierarchically structured graph neural architecture which learns not only the local region-wise geographical dependencies, but also the spatial semantics from a global perspective. Experiments on several real-life traffic datasets demonstrate that ST-GDN outperforms different types of state-of-the-art baselines.
  arXiv  Detail & Related papers  (2021-10-08T11:19:06Z)
• MAF-GNN: Multi-adaptive Spatiotemporal-flow Graph Neural Network for Traffic Speed Forecasting [3.614768552081925]
  We propose a Multi-adaptive Spatiotemporal-flow Graph Neural Network (MAF-GNN) for traffic speed forecasting. MAF-GNN introduces an effective Multi-adaptive Adjacency Matrices Mechanism to capture multiple latent spatial dependencies between traffic nodes. It achieves better performance than other models on two real-world datasets of public traffic network, METR-LA and PeMS-Bay.
  arXiv  Detail & Related papers  (2021-08-08T09:06:43Z)
• SST-GNN: Simplified Spatio-temporal Traffic forecasting model using Graph Neural Network [2.524966118517392]
  We have designed a simplified S-temporal GNN(SST-GNN) that effectively encodes the dependency by separately aggregating different neighborhood. We have shown that our model has significantly outperformed the state-of-the-art models on three real-world traffic datasets.
  arXiv  Detail & Related papers  (2021-03-31T18:28:44Z)
• Multi-intersection Traffic Optimisation: A Benchmark Dataset and a Strong Baseline [85.9210953301628]
  Control of traffic signals is fundamental and critical to alleviate traffic congestion in urban areas. Because of the high complexity of modelling the problem, experimental settings of current works are often inconsistent. We propose a novel and strong baseline model based on deep reinforcement learning with the encoder-decoder structure.
  arXiv  Detail & Related papers  (2021-01-24T03:55:39Z)
• Spatial-Temporal Transformer Networks for Traffic Flow Forecasting [74.76852538940746]
  We propose a novel paradigm of Spatial-Temporal Transformer Networks (STTNs) to improve the accuracy of long-term traffic forecasting. Specifically, we present a new variant of graph neural networks, named spatial transformer, by dynamically modeling directed spatial dependencies. The proposed model enables fast and scalable training over a long range spatial-temporal dependencies.
  arXiv  Detail & Related papers  (2020-01-09T10:21:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.