$GRU^{spa}$: Gated Recurrent Unit with Spatial Attention for Spatio-Temporal Disaggregation

• URL: http://arxiv.org/abs/2306.07292v3
• Date: Tue, 19 Mar 2024 20:03:20 GMT
• Title: $GRU^{spa}$: Gated Recurrent Unit with Spatial Attention for Spatio-Temporal Disaggregation
• Authors: Bin Han, Bill Howe,
• Abstract summary: We consider models to complicate mobility-temporal data from a low-resolution, irregular partition to a high-resolution, irregular partition. We propose a model, Gated Recurrent Unit with Spatial Attention ($GRUspa$), where spatial attention layers are integrated into the original Gated Recurrent Unit (GRU) model. We show that $GRUspa$ provides a significant improvement over other neural models as well as typical methods.
• Score: 8.636014676778682
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Open data is frequently released spatially aggregated, usually to comply with privacy policies. But coarse, heterogeneous aggregations complicate learning and integration for downstream AI/ML systems. In this work, we consider models to disaggregate spatio-temporal data from a low-resolution, irregular partition (e.g., census tract) to a high-resolution, irregular partition (e.g., city block). We propose a model, Gated Recurrent Unit with Spatial Attention ($GRU^{spa}$), where spatial attention layers are integrated into the original Gated Recurrent Unit (GRU) model. The spatial attention layers capture spatial interactions among regions, while the gated recurrent module captures the temporal dependencies. Additionally, we utilize containment relationships between different geographic levels (e.g., when a given city block is wholly contained in a given census tract) to constrain the spatial attention layers. For situations where limited historical training data is available, we study transfer learning scenarios and show that a model pre-trained on one city variable can be fine-tuned for another city variable using only a few hundred samples. Evaluating these techniques on two mobility datasets, we find that $GRU^{spa}$ provides a significant improvement over other neural models as well as typical heuristic methods, allowing us to synthesize realistic point data over small regions useful for training downstream models.

Related papers

• Spatial-temporal Forecasting for Regions without Observations [13.805203053973772]
  We study spatial-temporal forecasting for a region of interest without any historical observations. We propose a model named STSM for the task. Our key insight is to learn from the locations that resemble those in the region of interest.
  arXiv  Detail & Related papers  (2024-01-19T06:26:05Z)
• PASTA: PArallel Spatio-Temporal Attention with spatial auto-correlation gating for fine-grained crowd flow prediction [33.08230699138568]
  We introduce a neural network named PArallel Spatio with spatial auto-correlation gating. The components in our approach include spatial auto-correlation gating, multi-scale residual block, and temporal attention gating module.
  arXiv  Detail & Related papers  (2023-10-02T14:10:42Z)
• Adaptive Hierarchical SpatioTemporal Network for Traffic Forecasting [70.66710698485745]
  We propose an Adaptive Hierarchical SpatioTemporal Network (AHSTN) to promote traffic forecasting. AHSTN exploits the spatial hierarchy and modeling multi-scale spatial correlations. Experiments on two real-world datasets show that AHSTN achieves better performance over several strong baselines.
  arXiv  Detail & Related papers  (2023-06-15T14:50:27Z)
• Geo-Adaptive Deep Spatio-Temporal predictive modeling for human mobility [5.864710987890994]
  Deep GA-vLS assumes data to be of fixed and regular shaped tensor shaped and face challenges of handling irregular data. We present a novel geo-aware enabled learning operation based on a novel data structure for dependencies while maintaining the recurrent mechanism.
  arXiv  Detail & Related papers  (2022-11-27T16:51:28Z)
• Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning [112.69497636932955]
  Federated learning aims to train models across different clients without the sharing of data for privacy considerations. We study how data heterogeneity affects the representations of the globally aggregated models. We propose sc FedDecorr, a novel method that can effectively mitigate dimensional collapse in federated learning.
  arXiv  Detail & Related papers  (2022-10-01T09:04:17Z)
• Semi-signed neural fitting for surface reconstruction from unoriented point clouds [53.379712818791894]
  We propose SSN-Fitting to reconstruct a better signed distance field. SSN-Fitting consists of a semi-signed supervision and a loss-based region sampling strategy. We conduct experiments to demonstrate that SSN-Fitting achieves state-of-the-art performance under different settings.
  arXiv  Detail & Related papers  (2022-06-14T09:40:17Z)
• CHALLENGER: Training with Attribution Maps [63.736435657236505]
  We show that utilizing attribution maps for training neural networks can improve regularization of models and thus increase performance. In particular, we show that our generic domain-independent approach yields state-of-the-art results in vision, natural language processing and on time series tasks.
  arXiv  Detail & Related papers  (2022-05-30T13:34:46Z)
• Unraveled Multilevel Transformation Networks for Predicting Sparsely-Observed Spatiotemporal Dynamics [12.627823168264209]
  We propose a model that learns to predict unknown dynamics using data from sparsely-distributed data sites. We demonstrate the advantage of our approach using both synthetic and real-world climate data.
  arXiv  Detail & Related papers  (2022-03-16T14:44:05Z)
• Contrastive Neighborhood Alignment [81.65103777329874]
  We present Contrastive Neighborhood Alignment (CNA), a manifold learning approach to maintain the topology of learned features. The target model aims to mimic the local structure of the source representation space using a contrastive loss. CNA is illustrated in three scenarios: manifold learning, where the model maintains the local topology of the original data in a dimension-reduced space; model distillation, where a small student model is trained to mimic a larger teacher; and legacy model update, where an older model is replaced by a more powerful one.
  arXiv  Detail & Related papers  (2022-01-06T04:58:31Z)
• Smoothing the Generative Latent Space with Mixup-based Distance Learning [32.838539968751924]
  We consider the situation where neither large scale dataset of our interest nor transferable source dataset is available. We propose latent mixup-based distance regularization on the feature space of both a generator and the counterpart discriminator.
  arXiv  Detail & Related papers  (2021-11-23T06:39:50Z)
• Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks [91.65637773358347]
  We propose a general graph neural network framework designed specifically for multivariate time series data. Our approach automatically extracts the uni-directed relations among variables through a graph learning module. Our proposed model outperforms the state-of-the-art baseline methods on 3 of 4 benchmark datasets.
  arXiv  Detail & Related papers  (2020-05-24T04:02:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.