Language Model Empowered Spatio-Temporal Forecasting via Physics-Aware Reprogramming

• URL: http://arxiv.org/abs/2408.14505v2
• Date: Fri, 4 Oct 2024 17:08:17 GMT
• Title: Language Model Empowered Spatio-Temporal Forecasting via Physics-Aware Reprogramming
• Authors: Hao Wang, Jindong Han, Wei Fan, Hao Liu,
• Abstract summary: We aim to harness the reasoning and generalization abilities of Pre-trained Language Models (PLMs) for intricate-temporal forecasting. We propose RePST, a physics-aware PLM reprogramming framework tailored fortemporal forecasting. We show that the proposed RePST outperforms twelve state-of-the-art baseline methods, particularly in data-scarce scenarios.
• Score: 13.744891561921197
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Spatio-temporal forecasting is pivotal in numerous real-world applications, including transportation planning, energy management, and climate monitoring. In this work, we aim to harness the reasoning and generalization abilities of Pre-trained Language Models (PLMs) for more effective spatio-temporal forecasting, particularly in data-scarce scenarios. However, recent studies uncover that PLMs, which are primarily trained on textual data, often falter when tasked with modeling the intricate correlations in numerical time series, thereby limiting their effectiveness in comprehending spatio-temporal data. To bridge the gap, we propose RePST, a physics-aware PLM reprogramming framework tailored for spatio-temporal forecasting. Specifically, we first propose a physics-aware decomposer that adaptively disentangles spatially correlated time series into interpretable sub-components, which facilitates PLM to understand sophisticated spatio-temporal dynamics via a divide-and-conquer strategy. Moreover, we propose a selective discrete reprogramming scheme, which introduces an expanded spatio-temporal vocabulary space to project spatio-temporal series into discrete representations. This scheme minimizes the information loss during reprogramming and enriches the representations derived by PLMs. Extensive experiments on real-world datasets show that the proposed RePST outperforms twelve state-of-the-art baseline methods, particularly in data-scarce scenarios, highlighting the effectiveness and superior generalization capabilities of PLMs for spatio-temporal forecasting.

Related papers

• Physics-guided Active Sample Reweighting for Urban Flow Prediction [75.24539704456791]
  Urban flow prediction is a nuanced-temporal modeling that estimates the throughput of transportation services like buses, taxis and ride-driven models. Some recent prediction solutions bring remedies with the notion of physics-guided machine learning (PGML) We develop a atized physics-guided network (PN), and propose a data-aware framework Physics-guided Active Sample Reweighting (P-GASR)
  arXiv  Detail & Related papers  (2024-07-18T15:44:23Z)
• STD-PLM: Understanding Both Spatial and Temporal Properties of Spatial-Temporal Data with PLM [18.56267873980915]
  STD-PLM is capable of implementing both spatial-temporal forecasting and imputation tasks. STD-PLM understands spatial-temporal correlations via explicitly designed spatial and temporal tokenizers. STD-PLM exhibits competitive performance and generalization capabilities across the forecasting and imputation tasks.
  arXiv  Detail & Related papers  (2024-07-12T08:48:16Z)
• UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Prediction [26.69233687863233]
  Urban-temporal prediction is crucial for informed decision-making, such as traffic management, resource optimization, emergence response. We introduce UniST, a universal model designed for general urban-temporal prediction across wide range of scenarios by large language models.
  arXiv  Detail & Related papers  (2024-02-19T05:04:11Z)
• GATGPT: A Pre-trained Large Language Model with Graph Attention Network for Spatiotemporal Imputation [19.371155159744934]
  In real-world settings, such data often contain missing elements due to issues like sensor malfunctions and data transmission errors. The objective oftemporal imputation is to estimate these missing values by understanding the inherent spatial and temporal relationships in the observed time series. Traditionally, intricatetemporal imputation has relied on specific architectures, which suffer from limited applicability and high computational complexity. In contrast our approach integrates pre-trained large language models (LLMs) into intricatetemporal imputation, introducing a groundbreaking framework, GATGPT.
  arXiv  Detail & Related papers  (2023-11-24T08:15:11Z)
• Revisiting the Temporal Modeling in Spatio-Temporal Predictive Learning under A Unified View [73.73667848619343]
  We introduce USTEP (Unified S-TEmporal Predictive learning), an innovative framework that reconciles the recurrent-based and recurrent-free methods by integrating both micro-temporal and macro-temporal scales.
  arXiv  Detail & Related papers  (2023-10-09T16:17:42Z)
• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models [110.20279343734548]
  Time series forecasting holds significant importance in many real-world dynamic systems. We present Time-LLM, a reprogramming framework to repurpose large language models for time series forecasting. Time-LLM is a powerful time series learner that outperforms state-of-the-art, specialized forecasting models.
  arXiv  Detail & Related papers  (2023-10-03T01:31:25Z)
• MPR-Net:Multi-Scale Pattern Reproduction Guided Universality Time Series Interpretable Forecasting [13.790498420659636]
  Time series forecasting has received wide interest from existing research due to its broad applications inherent challenging. This paper proposes a forecasting model, MPR-Net. It first adaptively decomposes multi-scale historical series patterns using convolution operation, then constructs a pattern extension forecasting method based on the prior knowledge of pattern reproduction, and finally reconstructs future patterns into future series using deconvolution operation. By leveraging the temporal dependencies present in the time series, MPR-Net not only achieves linear time complexity, but also makes the forecasting process interpretable.
  arXiv  Detail & Related papers  (2023-07-13T13:16:01Z)
• OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning [67.07363529640784]
  We propose OpenSTL to categorize prevalent approaches into recurrent-based and recurrent-free models. We conduct standard evaluations on datasets across various domains, including synthetic moving object trajectory, human motion, driving scenes, traffic flow and forecasting weather. We find that recurrent-free models achieve a good balance between efficiency and performance than recurrent models.
  arXiv  Detail & Related papers  (2023-06-20T03:02:14Z)
• Temporal Predictive Coding For Model-Based Planning In Latent Space [80.99554006174093]
  We present an information-theoretic approach that employs temporal predictive coding to encode elements in the environment that can be predicted across time. We evaluate our model on a challenging modification of standard DMControl tasks where the background is replaced with natural videos that contain complex but irrelevant information to the planning task.
  arXiv  Detail & Related papers  (2021-06-14T04:31:15Z)
• Stochastically forced ensemble dynamic mode decomposition for forecasting and analysis of near-periodic systems [65.44033635330604]
  We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system. We show that its use of intrinsic linear dynamics offers a number of desirable properties in terms of interpretability and parsimony. Results are presented for a test case using load data from an electrical grid.
  arXiv  Detail & Related papers  (2020-10-08T20:25:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.