UrbanGPT: Spatio-Temporal Large Language Models

• URL: http://arxiv.org/abs/2403.00813v3
• Date: Sun, 19 May 2024 01:58:03 GMT
• Title: UrbanGPT: Spatio-Temporal Large Language Models
• Authors: Zhonghang Li, Lianghao Xia, Jiabin Tang, Yong Xu, Lei Shi, Long Xia, Dawei Yin, Chao Huang,
• Abstract summary: We present the UrbanPT, which seamlessly integrates atemporal-temporal encoder with instruction-tuning paradigm. We conduct extensive experiments on various public datasets, covering differenttemporal prediction tasks. The results consistently demonstrate that our UrbanPT, with its carefully designed architecture, consistently outperforms state-of-the-art baselines.
• Score: 34.79169613947957
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spatio-temporal prediction aims to forecast and gain insights into the ever-changing dynamics of urban environments across both time and space. Its purpose is to anticipate future patterns, trends, and events in diverse facets of urban life, including transportation, population movement, and crime rates. Although numerous efforts have been dedicated to developing neural network techniques for accurate predictions on spatio-temporal data, it is important to note that many of these methods heavily depend on having sufficient labeled data to generate precise spatio-temporal representations. Unfortunately, the issue of data scarcity is pervasive in practical urban sensing scenarios. Consequently, it becomes necessary to build a spatio-temporal model with strong generalization capabilities across diverse spatio-temporal learning scenarios. Taking inspiration from the remarkable achievements of large language models (LLMs), our objective is to create a spatio-temporal LLM that can exhibit exceptional generalization capabilities across a wide range of downstream urban tasks. To achieve this objective, we present the UrbanGPT, which seamlessly integrates a spatio-temporal dependency encoder with the instruction-tuning paradigm. This integration enables LLMs to comprehend the complex inter-dependencies across time and space, facilitating more comprehensive and accurate predictions under data scarcity. To validate the effectiveness of our approach, we conduct extensive experiments on various public datasets, covering different spatio-temporal prediction tasks. The results consistently demonstrate that our UrbanGPT, with its carefully designed architecture, consistently outperforms state-of-the-art baselines. These findings highlight the potential of building large language models for spatio-temporal learning, particularly in zero-shot scenarios where labeled data is scarce.

Related papers

• EasyST: A Simple Framework for Spatio-Temporal Prediction [18.291117879544945]
  We propose a simple framework for spatial-temporal prediction - EasyST paradigm. It learns lightweight and robust Multi-Layer Perceptrons (MLPs) generalization by distilling knowledge from complex-temporal GNNs. EasyST surpasses state-of-the-art approaches in terms of efficiency and accuracy.
  arXiv  Detail & Related papers  (2024-09-10T11:40:01Z)
• SMA-Hyper: Spatiotemporal Multi-View Fusion Hypergraph Learning for Traffic Accident Prediction [2.807532512532818]
  Current data-driven models often struggle with data sparsity and the integration of diverse urban data sources. We introduce a deep dynamic learning framework designed for traffic accident prediction. It incorporates dual adaptive graph learning mechanisms that enable high-order cross-regional learning. It also employs an advance attention mechanism to fuse multiple views of accident data and urban functional features.
  arXiv  Detail & Related papers  (2024-07-24T21:10:34Z)
• Physics-guided Active Sample Reweighting for Urban Flow Prediction [75.24539704456791]
  Urban flow prediction is a nuanced-temporal modeling that estimates the throughput of transportation services like buses, taxis and ride-driven models. Some recent prediction solutions bring remedies with the notion of physics-guided machine learning (PGML) We develop a atized physics-guided network (PN), and propose a data-aware framework Physics-guided Active Sample Reweighting (P-GASR)
  arXiv  Detail & Related papers  (2024-07-18T15:44:23Z)
• UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Prediction [26.69233687863233]
  Urban-temporal prediction is crucial for informed decision-making, such as traffic management, resource optimization, emergence response. We introduce UniST, a universal model designed for general urban-temporal prediction across wide range of scenarios by large language models.
  arXiv  Detail & Related papers  (2024-02-19T05:04:11Z)
• Rethinking Urban Mobility Prediction: A Super-Multivariate Time Series Forecasting Approach [71.67506068703314]
  Long-term urban mobility predictions play a crucial role in the effective management of urban facilities and services. Traditionally, urban mobility data has been structured as videos, treating longitude and latitude as fundamental pixels. In our research, we introduce a fresh perspective on urban mobility prediction. Instead of oversimplifying urban mobility data as traditional video data, we regard it as a complex time series.
  arXiv  Detail & Related papers  (2023-12-04T07:39:05Z)
• Unified Data Management and Comprehensive Performance Evaluation for Urban Spatial-Temporal Prediction [Experiment, Analysis & Benchmark] [78.05103666987655]
  This work addresses challenges in accessing and utilizing diverse urban spatial-temporal datasets. We introduceatomic files, a unified storage format designed for urban spatial-temporal big data, and validate its effectiveness on 40 diverse datasets. We conduct extensive experiments using diverse models and datasets, establishing a performance leaderboard and identifying promising research directions.
  arXiv  Detail & Related papers  (2023-08-24T16:20:00Z)
• Learning Dynamic Graphs from All Contextual Information for Accurate Point-of-Interest Visit Forecasting [9.670949636600035]
  Busyness Graph Neural Network (BysGNN) is a temporal graph neural network designed to learn and uncover the underlying multi-context correlations. By incorporating all contextual, temporal, and spatial signals, we observe a significant improvement in our forecasting accuracy over state-of-the-art forecasting models.
  arXiv  Detail & Related papers  (2023-06-28T05:14:03Z)
• LibCity: A Unified Library Towards Efficient and Comprehensive Urban Spatial-Temporal Prediction [74.08181247675095]
  There are limitations in the existing field, including open-source data being in various formats and difficult to use. We propose LibCity, an open-source library that offers researchers a credible experimental tool and a convenient development framework.
  arXiv  Detail & Related papers  (2023-04-27T17:19:26Z)
• Predicting Future Occupancy Grids in Dynamic Environment with Spatio-Temporal Learning [63.25627328308978]
  We propose a-temporal prediction network pipeline to generate future occupancy predictions. Compared to current SOTA, our approach predicts occupancy for a longer horizon of 3 seconds. We publicly release our grid occupancy dataset based on nulis to support further research.
  arXiv  Detail & Related papers  (2022-05-06T13:45:32Z)
• Temporal Predictive Coding For Model-Based Planning In Latent Space [80.99554006174093]
  We present an information-theoretic approach that employs temporal predictive coding to encode elements in the environment that can be predicted across time. We evaluate our model on a challenging modification of standard DMControl tasks where the background is replaced with natural videos that contain complex but irrelevant information to the planning task.
  arXiv  Detail & Related papers  (2021-06-14T04:31:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.