Rethinking Spatio-Temporal Transformer for Traffic Prediction:Multi-level Multi-view Augmented Learning Framework

• URL: http://arxiv.org/abs/2406.11921v1
• Date: Mon, 17 Jun 2024 07:36:57 GMT
• Title: Rethinking Spatio-Temporal Transformer for Traffic Prediction:Multi-level Multi-view Augmented Learning Framework
• Authors: Jiaqi Lin, Qianqian Ren,
• Abstract summary: Traffic is a challenging-temporal forecasting problem that involves highly complex semantic correlations. This paper proposes a Multi-level Multi-view Augmented-temporal Transformer (LVST) for traffic prediction.
• Score: 4.773547922851949
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traffic prediction is a challenging spatio-temporal forecasting problem that involves highly complex spatio-temporal correlations. This paper proposes a Multi-level Multi-view Augmented Spatio-temporal Transformer (LVSTformer) for traffic prediction. The model aims to capture spatial dependencies from three different levels: local geographic, global semantic, and pivotal nodes, along with long- and short-term temporal dependencies. Specifically, we design three spatial augmented views to delve into the spatial information from the perspectives of local, global, and pivotal nodes. By combining three spatial augmented views with three parallel spatial self-attention mechanisms, the model can comprehensively captures spatial dependencies at different levels. We design a gated temporal self-attention mechanism to effectively capture long- and short-term temporal dependencies. Furthermore, a spatio-temporal context broadcasting module is introduced between two spatio-temporal layers to ensure a well-distributed allocation of attention scores, alleviating overfitting and information loss, and enhancing the generalization ability and robustness of the model. A comprehensive set of experiments is conducted on six well-known traffic benchmarks, the experimental results demonstrate that LVSTformer achieves state-of-the-art performance compared to competing baselines, with the maximum improvement reaching up to 4.32%.

Related papers

• Hybrid Transformer and Spatial-Temporal Self-Supervised Learning for Long-term Traffic Prediction [1.8531577178922987]
  We propose a model that combines hybrid Transformer and self-supervised learning. The model enhances its adaptive data augmentation by applying data augmentation techniques at the sequence-level of the traffic. We design two self-supervised learning tasks to model the temporal and spatial dependencies, thereby improving the accuracy and ability of the model.
  arXiv  Detail & Related papers  (2024-01-29T06:17:23Z)
• Rethinking Urban Mobility Prediction: A Super-Multivariate Time Series Forecasting Approach [71.67506068703314]
  Long-term urban mobility predictions play a crucial role in the effective management of urban facilities and services. Traditionally, urban mobility data has been structured as videos, treating longitude and latitude as fundamental pixels. In our research, we introduce a fresh perspective on urban mobility prediction. Instead of oversimplifying urban mobility data as traditional video data, we regard it as a complex time series.
  arXiv  Detail & Related papers  (2023-12-04T07:39:05Z)
• Triplet Attention Transformer for Spatiotemporal Predictive Learning [9.059462850026216]
  We propose an innovative triplet attention transformer designed to capture both inter-frame dynamics and intra-frame static features. The model incorporates the Triplet Attention Module (TAM), which replaces traditional recurrent units by exploring self-attention mechanisms in temporal, spatial, and channel dimensions.
  arXiv  Detail & Related papers  (2023-10-28T12:49:33Z)
• Multi-Scale Spatial-Temporal Recurrent Networks for Traffic Flow Prediction [13.426775574655135]
  We propose a Multi-Scale Spatial-Temporal Recurrent Network for traffic flow prediction, namely MSSTRN. We propose a spatial-temporal synchronous attention mechanism that integrates adaptive position graph convolutions into the self-attention mechanism to achieve synchronous capture of spatial-temporal dependencies. Our model achieves the best prediction accuracy with non-trivial margins compared to all the twenty baseline methods.
  arXiv  Detail & Related papers  (2023-10-12T08:52:36Z)
• Adaptive Hierarchical SpatioTemporal Network for Traffic Forecasting [70.66710698485745]
  We propose an Adaptive Hierarchical SpatioTemporal Network (AHSTN) to promote traffic forecasting. AHSTN exploits the spatial hierarchy and modeling multi-scale spatial correlations. Experiments on two real-world datasets show that AHSTN achieves better performance over several strong baselines.
  arXiv  Detail & Related papers  (2023-06-15T14:50:27Z)
• Global-to-Local Modeling for Video-based 3D Human Pose and Shape Estimation [53.04781510348416]
  Video-based 3D human pose and shape estimations are evaluated by intra-frame accuracy and inter-frame smoothness. We propose to structurally decouple the modeling of long-term and short-term correlations in an end-to-end framework, Global-to-Local Transformer (GLoT) Our GLoT surpasses previous state-of-the-art methods with the lowest model parameters on popular benchmarks, i.e., 3DPW, MPI-INF-3DHP, and Human3.6M.
  arXiv  Detail & Related papers  (2023-03-26T14:57:49Z)
• Attention-based Spatial-Temporal Graph Convolutional Recurrent Networks for Traffic Forecasting [12.568905377581647]
  Traffic forecasting is one of the most fundamental problems in transportation science and artificial intelligence. Existing methods cannot accurately model both long-term and short-term temporal correlations simultaneously. We propose a novel spatial-temporal neural network framework, which consists of a graph convolutional recurrent module (GCRN) and a global attention module.
  arXiv  Detail & Related papers  (2023-02-25T03:37:00Z)
• PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Prediction [78.05103666987655]
  spatial-temporal Graph Neural Network (GNN) models have emerged as one of the most promising methods to solve this problem. We propose a novel propagation delay-aware dynamic long-range transFormer, namely PDFormer, for accurate traffic flow prediction. Our method can not only achieve state-of-the-art performance but also exhibit competitive computational efficiency.
  arXiv  Detail & Related papers  (2023-01-19T08:42:40Z)
• STJLA: A Multi-Context Aware Spatio-Temporal Joint Linear Attention Network for Traffic Forecasting [7.232141271583618]
  We propose a novel deep learning model for traffic forecasting named inefficient-Context Spatio-Temporal Joint Linear Attention (SSTLA) SSTLA applies linear attention to a joint graph to capture global dependence between alltemporal- nodes efficiently. Experiments on two real-world traffic datasets, England and Temporal7, demonstrate that our STJLA can achieve 9.83% and 3.08% 3.08% accuracy in MAE measure over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2021-12-04T06:39:18Z)
• A Spatial-Temporal Attentive Network with Spatial Continuity for Trajectory Prediction [74.00750936752418]
  We propose a novel model named spatial-temporal attentive network with spatial continuity (STAN-SC) First, spatial-temporal attention mechanism is presented to explore the most useful and important information. Second, we conduct a joint feature sequence based on the sequence and instant state information to make the generative trajectories keep spatial continuity.
  arXiv  Detail & Related papers  (2020-03-13T04:35:50Z)
• Spatial-Temporal Transformer Networks for Traffic Flow Forecasting [74.76852538940746]
  We propose a novel paradigm of Spatial-Temporal Transformer Networks (STTNs) to improve the accuracy of long-term traffic forecasting. Specifically, we present a new variant of graph neural networks, named spatial transformer, by dynamically modeling directed spatial dependencies. The proposed model enables fast and scalable training over a long range spatial-temporal dependencies.
  arXiv  Detail & Related papers  (2020-01-09T10:21:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.