Temporal Graph MLP Mixer for Spatio-Temporal Forecasting

• URL: http://arxiv.org/abs/2501.10214v1
• Date: Fri, 17 Jan 2025 14:13:48 GMT
• Title: Temporal Graph MLP Mixer for Spatio-Temporal Forecasting
• Authors: Muhammad Bilal, Luis Carretero Lopez,
• Abstract summary: Temporal Graph-Mixer (T-GMM) architecture designed to address missing data challenges. Model combines node-level processing with subgraph encoding to capture localized spatial dependencies. Experiments on AQI, ENGRAD, PV-US and METR-LA datasets demonstrate model's ability to effectively forecast even in the presence of significant missing data.
• Score: 1.5696662871407674
• License:
• Abstract: Spatiotemporal forecasting is critical in applications such as traffic prediction, climate modeling, and environmental monitoring. However, the prevalence of missing data in real-world sensor networks significantly complicates this task. In this paper, we introduce the Temporal Graph MLP-Mixer (T-GMM), a novel architecture designed to address these challenges. The model combines node-level processing with patch-level subgraph encoding to capture localized spatial dependencies while leveraging a three-dimensional MLP-Mixer to handle temporal, spatial, and feature-based dependencies. Experiments on the AQI, ENGRAD, PV-US and METR-LA datasets demonstrate the model's ability to effectively forecast even in the presence of significant missing data. While not surpassing state-of-the-art models in all scenarios, the T-GMM exhibits strong learning capabilities, particularly in capturing long-range dependencies. These results highlight its potential for robust, scalable spatiotemporal forecasting.

Related papers

• PreMixer: MLP-Based Pre-training Enhanced MLP-Mixers for Large-scale Traffic Forecasting [30.055634767677823]
  In urban computing, precise and swift forecasting of time series data from traffic networks is crucial. Current research limitations because of inherent inefficiency of model and their unsuitability for large-scale traffic applications due to model complexity. This paper proposes a novel framework, named PreMixer, designed to bridge this gap. It features a predictive model and a pre-training mechanism, both based on the principles of Multi-Layer Perceptrons (MLP) Our framework achieves comparable state-of-theart performance while maintaining high computational efficiency, as verified by extensive experiments on large-scale traffic datasets.
  arXiv  Detail & Related papers  (2024-12-18T08:35:40Z)
• ST-FiT: Inductive Spatial-Temporal Forecasting with Limited Training Data [59.78770412981611]
  In real-world applications, most nodes may not possess any available temporal data during training. We propose a principled framework named ST-FiT to handle this problem.
  arXiv  Detail & Related papers  (2024-12-14T17:51:29Z)
• System States Forecasting of Microservices with Dynamic Spatio-Temporal Data [9.519440926598524]
  Current forecasting methods are insufficient in environments where relationships are critical. In both short-term and long-term forecasting tasks, our model consistently achieved a 8.6% reduction in MAE(Mean Absolute Error) and a 2.2% reduction in MSE (Mean Squared Error)
  arXiv  Detail & Related papers  (2024-08-15T02:52:02Z)
• DyG-Mamba: Continuous State Space Modeling on Dynamic Graphs [59.434893231950205]
  Dynamic graph learning aims to uncover evolutionary laws in real-world systems. We propose DyG-Mamba, a new continuous state space model for dynamic graph learning. We show that DyG-Mamba achieves state-of-the-art performance on most datasets.
  arXiv  Detail & Related papers  (2024-08-13T15:21:46Z)
• SFANet: Spatial-Frequency Attention Network for Weather Forecasting [54.470205739015434]
  Weather forecasting plays a critical role in various sectors, driving decision-making and risk management. Traditional methods often struggle to capture the complex dynamics of meteorological systems. We propose a novel framework designed to address these challenges and enhance the accuracy of weather prediction.
  arXiv  Detail & Related papers  (2024-05-29T08:00:15Z)
• RPMixer: Shaking Up Time Series Forecasting with Random Projections for Large Spatial-Temporal Data [33.0546525587517]
  We propose a all-Multi-Layer Perceptron (all-MLP) time series forecasting architecture called RPMixer. Our method capitalizes on the ensemble-like behavior of deep neural networks, where each individual block behaves like a base learner in an ensemble model.
  arXiv  Detail & Related papers  (2024-02-16T07:28:59Z)
• Learning Robust Precipitation Forecaster by Temporal Frame Interpolation [65.5045412005064]
  We develop a robust precipitation forecasting model that demonstrates resilience against spatial-temporal discrepancies. Our approach has led to significant improvements in forecasting precision, culminating in our model securing textit1st place in the transfer learning leaderboard of the textitWeather4cast'23 competition.
  arXiv  Detail & Related papers  (2023-11-30T08:22:08Z)
• GATGPT: A Pre-trained Large Language Model with Graph Attention Network for Spatiotemporal Imputation [19.371155159744934]
  In real-world settings, such data often contain missing elements due to issues like sensor malfunctions and data transmission errors. The objective oftemporal imputation is to estimate these missing values by understanding the inherent spatial and temporal relationships in the observed time series. Traditionally, intricatetemporal imputation has relied on specific architectures, which suffer from limited applicability and high computational complexity. In contrast our approach integrates pre-trained large language models (LLMs) into intricatetemporal imputation, introducing a groundbreaking framework, GATGPT.
  arXiv  Detail & Related papers  (2023-11-24T08:15:11Z)
• Attention-based Spatial-Temporal Graph Convolutional Recurrent Networks for Traffic Forecasting [12.568905377581647]
  Traffic forecasting is one of the most fundamental problems in transportation science and artificial intelligence. Existing methods cannot accurately model both long-term and short-term temporal correlations simultaneously. We propose a novel spatial-temporal neural network framework, which consists of a graph convolutional recurrent module (GCRN) and a global attention module.
  arXiv  Detail & Related papers  (2023-02-25T03:37:00Z)
• Deep Autoregressive Models with Spectral Attention [74.08846528440024]
  We propose a forecasting architecture that combines deep autoregressive models with a Spectral Attention (SA) module. By characterizing in the spectral domain the embedding of the time series as occurrences of a random process, our method can identify global trends and seasonality patterns. Two spectral attention models, global and local to the time series, integrate this information within the forecast and perform spectral filtering to remove time series's noise.
  arXiv  Detail & Related papers  (2021-07-13T11:08:47Z)
• Federated Learning in the Sky: Aerial-Ground Air Quality Sensing Framework with UAV Swarms [53.38353133198842]
  Air quality significantly affects human health, it is increasingly important to accurately and timely predict the Air Quality Index (AQI) This paper proposes a new federated learning-based aerial-ground air quality sensing framework for fine-grained 3D air quality monitoring and forecasting. For ground sensing systems, we propose a Graph Convolutional neural network-based Long Short-Term Memory (GC-LSTM) model to achieve accurate, real-time and future AQI inference.
  arXiv  Detail & Related papers  (2020-07-23T13:32:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.