UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Prediction
- URL: http://arxiv.org/abs/2402.11838v5
- Date: Mon, 1 Jul 2024 02:51:58 GMT
- Title: UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Prediction
- Authors: Yuan Yuan, Jingtao Ding, Jie Feng, Depeng Jin, Yong Li,
- Abstract summary: Urban-temporal prediction is crucial for informed decision-making, such as traffic management, resource optimization, emergence response.
We introduce UniST, a universal model designed for general urban-temporal prediction across wide range of scenarios by large language models.
- Score: 26.69233687863233
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Urban spatio-temporal prediction is crucial for informed decision-making, such as traffic management, resource optimization, and emergence response. Despite remarkable breakthroughs in pretrained natural language models that enable one model to handle diverse tasks, a universal solution for spatio-temporal prediction remains challenging Existing prediction approaches are typically tailored for specific spatio-temporal scenarios, requiring task-specific model designs and extensive domain-specific training data. In this study, we introduce UniST, a universal model designed for general urban spatio-temporal prediction across a wide range of scenarios. Inspired by large language models, UniST achieves success through: (i) utilizing diverse spatio-temporal data from different scenarios, (ii) effective pre-training to capture complex spatio-temporal dynamics, (iii) knowledge-guided prompts to enhance generalization capabilities. These designs together unlock the potential of building a universal model for various scenarios Extensive experiments on more than 20 spatio-temporal scenarios demonstrate UniST's efficacy in advancing state-of-the-art performance, especially in few-shot and zero-shot prediction. The datasets and code implementation are released on https://github.com/tsinghua-fib-lab/UniST.
Related papers
- Physics-guided Active Sample Reweighting for Urban Flow Prediction [75.24539704456791]
Urban flow prediction is a nuanced-temporal modeling that estimates the throughput of transportation services like buses, taxis and ride-driven models.
Some recent prediction solutions bring remedies with the notion of physics-guided machine learning (PGML)
We develop a atized physics-guided network (PN), and propose a data-aware framework Physics-guided Active Sample Reweighting (P-GASR)
arXiv Detail & Related papers (2024-07-18T15:44:23Z) - Adaptive Prediction Ensemble: Improving Out-of-Distribution Generalization of Motion Forecasting [15.916325272109454]
We propose a novel framework, Adaptive Prediction Ensemble (APE), which integrates deep learning and rule-based prediction experts.
A learned routing function, trained concurrently with the deep learning model, dynamically selects the most reliable prediction based on the input scenario.
This work highlights the potential of hybrid approaches for robust and generalizable motion prediction in autonomous driving.
arXiv Detail & Related papers (2024-07-12T17:57:00Z) - FlashST: A Simple and Universal Prompt-Tuning Framework for Traffic Prediction [22.265095967530296]
FlashST is a framework that adapts pre-trained models to generalize specific characteristics of diverse datasets.
It captures a shift of pre-training and downstream data, facilitating effective adaptation to diverse scenarios.
Empirical evaluations demonstrate the effectiveness of FlashST across different scenarios.
arXiv Detail & Related papers (2024-05-28T07:18:52Z) - A Survey on Diffusion Models for Time Series and Spatio-Temporal Data [92.1255811066468]
We review the use of diffusion models in time series and S-temporal data, categorizing them by model, task type, data modality, and practical application domain.
We categorize diffusion models into unconditioned and conditioned types discuss time series and S-temporal data separately.
Our survey covers their application extensively in various fields including healthcare, recommendation, climate, energy, audio, and transportation.
arXiv Detail & Related papers (2024-04-29T17:19:40Z) - UrbanGPT: Spatio-Temporal Large Language Models [34.79169613947957]
We present the UrbanPT, which seamlessly integrates atemporal-temporal encoder with instruction-tuning paradigm.
We conduct extensive experiments on various public datasets, covering differenttemporal prediction tasks.
The results consistently demonstrate that our UrbanPT, with its carefully designed architecture, consistently outperforms state-of-the-art baselines.
arXiv Detail & Related papers (2024-02-25T12:37:29Z) - Unified Training of Universal Time Series Forecasting Transformers [104.56318980466742]
We present a Masked-based Universal Time Series Forecasting Transformer (Moirai)
Moirai is trained on our newly introduced Large-scale Open Time Series Archive (LOTSA) featuring over 27B observations across nine domains.
Moirai achieves competitive or superior performance as a zero-shot forecaster when compared to full-shot models.
arXiv Detail & Related papers (2024-02-04T20:00:45Z) - Towards Unifying Diffusion Models for Probabilistic Spatio-Temporal
Graph Learning [28.50648620744963]
Existing approaches tackle different learning tasks independently, tailoring their models to intrinsic uncertainties in the Web-temporal data.
We introduce Unified S-temporal Diffusion Models (USTD) to address the tasks uniformly within the uncertainty-aware diffusion framework.
USTD is designed comprising a shared-temporal encoder and attention-based denoising that networks are task-specific.
arXiv Detail & Related papers (2023-10-26T12:48:43Z) - TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting [24.834846119163885]
We propose a novel framework, TEMPO, that can effectively learn time series representations.
TEMPO expands the capability for dynamically modeling real-world temporal phenomena from data within diverse domains.
arXiv Detail & Related papers (2023-10-08T00:02:25Z) - OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive
Learning [67.07363529640784]
We propose OpenSTL to categorize prevalent approaches into recurrent-based and recurrent-free models.
We conduct standard evaluations on datasets across various domains, including synthetic moving object trajectory, human motion, driving scenes, traffic flow and forecasting weather.
We find that recurrent-free models achieve a good balance between efficiency and performance than recurrent models.
arXiv Detail & Related papers (2023-06-20T03:02:14Z) - Deep Autoregressive Models with Spectral Attention [74.08846528440024]
We propose a forecasting architecture that combines deep autoregressive models with a Spectral Attention (SA) module.
By characterizing in the spectral domain the embedding of the time series as occurrences of a random process, our method can identify global trends and seasonality patterns.
Two spectral attention models, global and local to the time series, integrate this information within the forecast and perform spectral filtering to remove time series's noise.
arXiv Detail & Related papers (2021-07-13T11:08:47Z) - Model-Attentive Ensemble Learning for Sequence Modeling [86.4785354333566]
We present Model-Attentive Ensemble learning for Sequence modeling (MAES)
MAES is a mixture of time-series experts which leverages an attention-based gating mechanism to specialize the experts on different sequence dynamics and adaptively weight their predictions.
We demonstrate that MAES significantly out-performs popular sequence models on datasets subject to temporal shift.
arXiv Detail & Related papers (2021-02-23T05:23:35Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.