TESTAM: A Time-Enhanced Spatio-Temporal Attention Model with Mixture of Experts

• URL: http://arxiv.org/abs/2403.02600v1
• Date: Tue, 5 Mar 2024 02:27:52 GMT
• Title: TESTAM: A Time-Enhanced Spatio-Temporal Attention Model with Mixture of Experts
• Authors: Hyunwook Lee, Sungahn Ko
• Abstract summary: We propose a novel deep learning model named TESTAM, which individually models recurring and non-recurring traffic patterns. We show that TESTAM achieves a better indication and modeling of recurring and non-recurring traffic.
• Score: 6.831798156287652
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate traffic forecasting is challenging due to the complex dependency on road networks, various types of roads, and the abrupt speed change due to the events. Recent works mainly focus on dynamic spatial modeling with adaptive graph embedding or graph attention having less consideration for temporal characteristics and in-situ modeling. In this paper, we propose a novel deep learning model named TESTAM, which individually models recurring and non-recurring traffic patterns by a mixture-of-experts model with three experts on temporal modeling, spatio-temporal modeling with static graph, and dynamic spatio-temporal dependency modeling with dynamic graph. By introducing different experts and properly routing them, TESTAM could better model various circumstances, including spatially isolated nodes, highly related nodes, and recurring and non-recurring events. For the proper routing, we reformulate a gating problem into a classification problem with pseudo labels. Experimental results on three public traffic network datasets, METR-LA, PEMS-BAY, and EXPY-TKY, demonstrate that TESTAM achieves a better indication and modeling of recurring and non-recurring traffic. We published the official code at https://github.com/HyunWookL/TESTAM

Related papers

• Improving Traffic Flow Predictions with SGCN-LSTM: A Hybrid Model for Spatial and Temporal Dependencies [55.2480439325792]
  This paper introduces the Signal-Enhanced Graph Convolutional Network Long Short Term Memory (SGCN-LSTM) model for predicting traffic speeds across road networks. Experiments on the PEMS-BAY road network traffic dataset demonstrate the SGCN-LSTM model's effectiveness.
  arXiv  Detail & Related papers  (2024-11-01T00:37:00Z)
• Trajectory Flow Matching with Applications to Clinical Time Series Modeling [77.58277281319253]
  Trajectory Flow Matching (TFM) trains a Neural SDE in a simulation-free manner, bypassing backpropagation through the dynamics. We demonstrate improved performance on three clinical time series datasets in terms of absolute performance and uncertainty prediction.
  arXiv  Detail & Related papers  (2024-10-28T15:54:50Z)
• Trajeglish: Traffic Modeling as Next-Token Prediction [67.28197954427638]
  A longstanding challenge for self-driving development is simulating dynamic driving scenarios seeded from recorded driving logs. We apply tools from discrete sequence modeling to model how vehicles, pedestrians and cyclists interact in driving scenarios. Our model tops the Sim Agents Benchmark, surpassing prior work along the realism meta metric by 3.3% and along the interaction metric by 9.9%.
  arXiv  Detail & Related papers  (2023-12-07T18:53:27Z)
• Dynamic Hypergraph Structure Learning for Traffic Flow Forecasting [35.0288931087826]
  Traffic flow forecasting aims to predict future traffic conditions on the basis of networks and traffic conditions in the past. The problem is typically solved by modeling complex-temporal correlations in traffic data using far-temporal neural networks (GNNs) Existing methods follow the paradigm of message passing that aggregates neighborhood information linearly. In this paper, we propose a model named Dynamic Hyper Structure Learning (DyHSL) for traffic flow prediction.
  arXiv  Detail & Related papers  (2023-09-21T12:44:55Z)
• Attention-based Spatial-Temporal Graph Convolutional Recurrent Networks for Traffic Forecasting [12.568905377581647]
  Traffic forecasting is one of the most fundamental problems in transportation science and artificial intelligence. Existing methods cannot accurately model both long-term and short-term temporal correlations simultaneously. We propose a novel spatial-temporal neural network framework, which consists of a graph convolutional recurrent module (GCRN) and a global attention module.
  arXiv  Detail & Related papers  (2023-02-25T03:37:00Z)
• Automated Dilated Spatio-Temporal Synchronous Graph Modeling for Traffic Prediction [1.6449390849183363]
  We propose an automated dilated-temporal synchronous graph network prediction named Auto-DSTS for traffic prediction. Specifically, we propose an automated dilated-temporal-temporal graph (Auto-DSTS) module to capture the short-term and long-term-temporal correlations. Our model can achieve about 10% improvements compared with the state-of-art methods.
  arXiv  Detail & Related papers  (2022-07-22T00:50:39Z)
• Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs [65.18780403244178]
  We propose a continuous model to forecast Multivariate Time series with dynamic Graph neural Ordinary Differential Equations (MTGODE) Specifically, we first abstract multivariate time series into dynamic graphs with time-evolving node features and unknown graph structures. Then, we design and solve a neural ODE to complement missing graph topologies and unify both spatial and temporal message passing.
  arXiv  Detail & Related papers  (2022-02-17T02:17:31Z)
• Spatio-Temporal Joint Graph Convolutional Networks for Traffic Forecasting [75.10017445699532]
  Recent have shifted their focus towards formulating traffic forecasting as atemporal graph modeling problem. We propose a novel approach for accurate traffic forecasting on road networks over multiple future time steps.
  arXiv  Detail & Related papers  (2021-11-25T08:45:14Z)
• Dynamic Spatiotemporal Graph Convolutional Neural Networks for Traffic Data Imputation with Complex Missing Patterns [3.9318191265352196]
  We propose a novel deep learning framework called Dynamic Spatio Graph Contemporal Networks (DSTG) to impute missing traffic data. We introduce a graph structure estimation technique to model the dynamic spatial dependencies real-time traffic information and road network structure. Our proposed model outperforms existing deep learning models in all kinds of missing scenarios and the graph structure estimation technique contributes to the model performance.
  arXiv  Detail & Related papers  (2021-09-17T05:47:17Z)
• SST-GNN: Simplified Spatio-temporal Traffic forecasting model using Graph Neural Network [2.524966118517392]
  We have designed a simplified S-temporal GNN(SST-GNN) that effectively encodes the dependency by separately aggregating different neighborhood. We have shown that our model has significantly outperformed the state-of-the-art models on three real-world traffic datasets.
  arXiv  Detail & Related papers  (2021-03-31T18:28:44Z)
• Spatial-Temporal Transformer Networks for Traffic Flow Forecasting [74.76852538940746]
  We propose a novel paradigm of Spatial-Temporal Transformer Networks (STTNs) to improve the accuracy of long-term traffic forecasting. Specifically, we present a new variant of graph neural networks, named spatial transformer, by dynamically modeling directed spatial dependencies. The proposed model enables fast and scalable training over a long range spatial-temporal dependencies.
  arXiv  Detail & Related papers  (2020-01-09T10:21:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.