OpenCity: Open Spatio-Temporal Foundation Models for Traffic Prediction

• URL: http://arxiv.org/abs/2408.10269v1
• Date: Fri, 16 Aug 2024 15:20:36 GMT
• Title: OpenCity: Open Spatio-Temporal Foundation Models for Traffic Prediction
• Authors: Zhonghang Li, Long Xia, Lei Shi, Yong Xu, Dawei Yin, Chao Huang,
• Abstract summary: We introduce a novel foundation model, named OpenCity, that can effectively capture and normalize the underlying unseen-temporal patterns from diverse data characteristics. OpenCity integrates the Transformer architecture with graph neural networks to model the complex-temporal dependencies in traffic data. Experimental results demonstrate that OpenCity exhibits exceptional zero-shot performance.
• Score: 29.514461050436932
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Accurate traffic forecasting is crucial for effective urban planning and transportation management, enabling efficient resource allocation and enhanced travel experiences. However, existing models often face limitations in generalization, struggling with zero-shot prediction on unseen regions and cities, as well as diminished long-term accuracy. This is primarily due to the inherent challenges in handling the spatial and temporal heterogeneity of traffic data, coupled with the significant distribution shift across time and space. In this work, we aim to unlock new possibilities for building versatile, resilient and adaptive spatio-temporal foundation models for traffic prediction. To achieve this goal, we introduce a novel foundation model, named OpenCity, that can effectively capture and normalize the underlying spatio-temporal patterns from diverse data characteristics, facilitating zero-shot generalization across diverse urban environments. OpenCity integrates the Transformer architecture with graph neural networks to model the complex spatio-temporal dependencies in traffic data. By pre-training OpenCity on large-scale, heterogeneous traffic datasets, we enable the model to learn rich, generalizable representations that can be seamlessly applied to a wide range of traffic forecasting scenarios. Experimental results demonstrate that OpenCity exhibits exceptional zero-shot predictive performance. Moreover, OpenCity showcases promising scaling laws, suggesting the potential for developing a truly one-for-all traffic prediction solution that can adapt to new urban contexts with minimal overhead. We made our proposed OpenCity model open-source and it is available at the following link: https://github.com/HKUDS/OpenCity.

Related papers

• Strada-LLM: Graph LLM for traffic prediction [62.2015839597764]
  A considerable challenge in traffic prediction lies in handling the diverse data distributions caused by vastly different traffic conditions. We propose a graph-aware LLM for traffic prediction that considers proximal traffic information. We adopt a lightweight approach for efficient domain adaptation when facing new data distributions in few-shot fashion.
  arXiv  Detail & Related papers  (2024-10-28T09:19:29Z)
• SSMT: Few-Shot Traffic Forecasting with Single Source Meta-Transfer [19.768107394061374]
  We introduce Single Source Meta-Transfer Learning (SSMT) which relies only on a single source city for traffic prediction. Our method harnesses this transferred knowledge to enable few-shot traffic forecasting, particularly when the target city possesses limited data. We extend the idea of sinusoidal positional encoding to establish meta-learning tasks by leveraging diverse temporal traffic patterns from the source city.
  arXiv  Detail & Related papers  (2024-10-21T02:17:25Z)
• Physics-guided Active Sample Reweighting for Urban Flow Prediction [75.24539704456791]
  Urban flow prediction is a nuanced-temporal modeling that estimates the throughput of transportation services like buses, taxis and ride-driven models. Some recent prediction solutions bring remedies with the notion of physics-guided machine learning (PGML) We develop a atized physics-guided network (PN), and propose a data-aware framework Physics-guided Active Sample Reweighting (P-GASR)
  arXiv  Detail & Related papers  (2024-07-18T15:44:23Z)
• Spatio-Temporal Few-Shot Learning via Diffusive Neural Network Generation [25.916891462152044]
  We propose a novel generative pre-training framework, GPD, for intricate few-shot learning with urban knowledge transfer. We recast a generative diffusion model, which generates tailored neural networks guided by prompts. GPD consistently outperforms state-of-the-art baselines on datasets for tasks such as traffic speed prediction and crowd flow prediction.
  arXiv  Detail & Related papers  (2024-02-19T08:11:26Z)
• Rethinking Urban Mobility Prediction: A Super-Multivariate Time Series Forecasting Approach [71.67506068703314]
  Long-term urban mobility predictions play a crucial role in the effective management of urban facilities and services. Traditionally, urban mobility data has been structured as videos, treating longitude and latitude as fundamental pixels. In our research, we introduce a fresh perspective on urban mobility prediction. Instead of oversimplifying urban mobility data as traditional video data, we regard it as a complex time series.
  arXiv  Detail & Related papers  (2023-12-04T07:39:05Z)
• LibCity: A Unified Library Towards Efficient and Comprehensive Urban Spatial-Temporal Prediction [74.08181247675095]
  There are limitations in the existing field, including open-source data being in various formats and difficult to use. We propose LibCity, an open-source library that offers researchers a credible experimental tool and a convenient development framework.
  arXiv  Detail & Related papers  (2023-04-27T17:19:26Z)
• PDFormer: Propagation Delay-Aware Dynamic Long-Range Transformer for Traffic Flow Prediction [78.05103666987655]
  spatial-temporal Graph Neural Network (GNN) models have emerged as one of the most promising methods to solve this problem. We propose a novel propagation delay-aware dynamic long-range transFormer, namely PDFormer, for accurate traffic flow prediction. Our method can not only achieve state-of-the-art performance but also exhibit competitive computational efficiency.
  arXiv  Detail & Related papers  (2023-01-19T08:42:40Z)
• Conditioned Human Trajectory Prediction using Iterative Attention Blocks [70.36888514074022]
  We present a simple yet effective pedestrian trajectory prediction model aimed at pedestrians positions prediction in urban-like environments. Our model is a neural-based architecture that can run several layers of attention blocks and transformers in an iterative sequential fashion. We show that without explicit introduction of social masks, dynamical models, social pooling layers, or complicated graph-like structures, it is possible to produce on par results with SoTA models.
  arXiv  Detail & Related papers  (2022-06-29T07:49:48Z)
• Few-Shot Traffic Prediction with Graph Networks using Locale as Relational Inductive Biases [7.173242326298134]
  In many cities, the available amount of traffic data is substantially below the minimum requirement due to the data collection expense. This paper develops a graph network (GN)-based deep learning model LocaleGn that depicts the traffic dynamics using localized data. It is also demonstrated that the learned knowledge from LocaleGn can be transferred across cities.
  arXiv  Detail & Related papers  (2022-03-08T09:46:50Z)
• Traffic4cast -- Large-scale Traffic Prediction using 3DResNet and Sparse-UNet [2.568084386350801]
  The aim is to build a machine learning model for predicting the normalized average traffic speed and flow of subregions of multiple-scale cities using historical data points. We explore 3DRparseNetes and Sparse-UNet approaches for the tasks in this competition. Our results show that both of the proposed models achieve much better performance than the baseline algorithms.
  arXiv  Detail & Related papers  (2021-11-10T23:40:52Z)
• TLab: Traffic Map Movie Forecasting Based on HR-NET [23.40323690536007]
  In our solution, hand-crafted features are input into the model in a form of channels. In terms of prediction accuracy, our solution has won 2nd place in NeurIPS 2020, Traffic4cast Challenge.
  arXiv  Detail & Related papers  (2020-11-13T18:48:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.