Self-Supervised Deconfounding Against Spatio-Temporal Shifts: Theory and
Modeling
- URL: http://arxiv.org/abs/2311.12472v2
- Date: Wed, 6 Mar 2024 12:57:27 GMT
- Title: Self-Supervised Deconfounding Against Spatio-Temporal Shifts: Theory and
Modeling
- Authors: Jiahao Ji, Wentao Zhang, Jingyuan Wang, Yue He and Chao Huang
- Abstract summary: In this work, we formalize the problem by constructing a causal graph of past traffic data, future traffic data, and external ST contexts.
We show that the failure of prior arts in OOD traffic data is due to ST contexts acting as a confounder, i.e., the common cause for past data and future ones.
We devise a Spatio-Temporal sElf-superVised dEconfounding (STEVE) framework to encode traffic data into two disentangled representations for associating invariant and variant ST contexts.
- Score: 48.09863133371918
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: As an important application of spatio-temporal (ST) data, ST traffic
forecasting plays a crucial role in improving urban travel efficiency and
promoting sustainable development. In practice, the dynamics of traffic data
frequently undergo distributional shifts attributed to external factors such as
time evolution and spatial differences. This entails forecasting models to
handle the out-of-distribution (OOD) issue where test data is distributed
differently from training data. In this work, we first formalize the problem by
constructing a causal graph of past traffic data, future traffic data, and
external ST contexts. We reveal that the failure of prior arts in OOD traffic
data is due to ST contexts acting as a confounder, i.e., the common cause for
past data and future ones. Then, we propose a theoretical solution named
Disentangled Contextual Adjustment (DCA) from a causal lens. It differentiates
invariant causal correlations against variant spurious ones and deconfounds the
effect of ST contexts. On top of that, we devise a Spatio-Temporal
sElf-superVised dEconfounding (STEVE) framework. It first encodes traffic data
into two disentangled representations for associating invariant and variant ST
contexts. Then, we use representative ST contexts from three conceptually
different perspectives (i.e., temporal, spatial, and semantic) as
self-supervised signals to inject context information into both
representations. In this way, we improve the generalization ability of the
learned context-oriented representations to OOD ST traffic forecasting.
Comprehensive experiments on four large-scale benchmark datasets demonstrate
that our STEVE consistently outperforms the state-of-the-art baselines across
various ST OOD scenarios.
Related papers
- Physics-guided Active Sample Reweighting for Urban Flow Prediction [75.24539704456791]
Urban flow prediction is a nuanced-temporal modeling that estimates the throughput of transportation services like buses, taxis and ride-driven models.
Some recent prediction solutions bring remedies with the notion of physics-guided machine learning (PGML)
We develop a atized physics-guided network (PN), and propose a data-aware framework Physics-guided Active Sample Reweighting (P-GASR)
arXiv Detail & Related papers (2024-07-18T15:44:23Z) - FlashST: A Simple and Universal Prompt-Tuning Framework for Traffic Prediction [22.265095967530296]
FlashST is a framework that adapts pre-trained models to generalize specific characteristics of diverse datasets.
It captures a shift of pre-training and downstream data, facilitating effective adaptation to diverse scenarios.
Empirical evaluations demonstrate the effectiveness of FlashST across different scenarios.
arXiv Detail & Related papers (2024-05-28T07:18:52Z) - Multi-Factor Spatio-Temporal Prediction based on Graph Decomposition
Learning [31.812810009108684]
We propose a multi-factor ST prediction task that predicts partial ST data evolution under different factors.
We instantiate a novel model-agnostic framework, named decomposition graph learning (STGDL) for multi-factor ST prediction.
Results show that our framework reduces prediction errors of various ST models by 9.41% on average.
arXiv Detail & Related papers (2023-10-16T13:12:27Z) - Spatio-Temporal Contrastive Self-Supervised Learning for POI-level Crowd
Flow Inference [23.8192952068949]
We present a novel Contrastive Self-learning framework for S-temporal data (CSST)
Our approach initiates with the construction of a spatial adjacency graph founded on the Points of Interest (POIs) and their respective distances.
We adopt a swapped prediction approach to anticipate the representation of the target subgraph from similar instances.
Our experiments, conducted on two real-world datasets, demonstrate that the CSST pre-trained on extensive noisy data consistently outperforms models trained from scratch.
arXiv Detail & Related papers (2023-09-06T02:51:24Z) - Spatio-Temporal Self-Supervised Learning for Traffic Flow Prediction [36.77135502344546]
We propose a novel Spatio-Supervised Learning (ST-SSL) traffic prediction framework.
Our ST-SSL is built over an integrated module with temporal spatial convolutions for encoding the information across space and time.
Experiments on four benchmark datasets demonstrate that ST-SSL consistently outperforms various state-of-the-art baselines.
arXiv Detail & Related papers (2022-12-07T10:02:01Z) - Averaging Spatio-temporal Signals using Optimal Transport and Soft
Alignments [110.79706180350507]
We show that our proposed loss can be used to define temporal-temporal baryechecenters as Fr'teche means duality.
Experiments on handwritten letters and brain imaging data confirm our theoretical findings.
arXiv Detail & Related papers (2022-03-11T09:46:22Z) - Handling Distribution Shifts on Graphs: An Invariance Perspective [77.14319095965058]
We formulate the OOD problem for node-level prediction on graphs.
We develop a new domain-invariant learning approach, named Explore-to-Extrapolate Risk Minimization.
We prove the validity of our method by theoretically showing its guarantee of a valid OOD solution.
arXiv Detail & Related papers (2022-02-05T02:31:01Z) - Towards Robust and Adaptive Motion Forecasting: A Causal Representation
Perspective [72.55093886515824]
We introduce a causal formalism of motion forecasting, which casts the problem as a dynamic process with three groups of latent variables.
We devise a modular architecture that factorizes the representations of invariant mechanisms and style confounders to approximate a causal graph.
Experiment results on synthetic and real datasets show that our three proposed components significantly improve the robustness and reusability of the learned motion representations.
arXiv Detail & Related papers (2021-11-29T18:59:09Z) - ProSTformer: Pre-trained Progressive Space-Time Self-attention Model for
Traffic Flow Forecasting [6.35012051925346]
Two issues prevent the approach from being effectively applied in traffic flow forecasting.
We first factor the dependencies and then a space-time self-attention mechanism named ProSTformer.
ProSTformer performs better or the same on the big scale datasets than six state-of-the-art methods by RMSE.
arXiv Detail & Related papers (2021-11-03T12:20:08Z) - Interpretable Time-series Representation Learning With Multi-Level
Disentanglement [56.38489708031278]
Disentangle Time Series (DTS) is a novel disentanglement enhancement framework for sequential data.
DTS generates hierarchical semantic concepts as the interpretable and disentangled representation of time-series.
DTS achieves superior performance in downstream applications, with high interpretability of semantic concepts.
arXiv Detail & Related papers (2021-05-17T22:02:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.