Preserving Dynamic Attention for Long-Term Spatial-Temporal Prediction

• URL: http://arxiv.org/abs/2006.08849v1
• Date: Tue, 16 Jun 2020 00:56:43 GMT
• Title: Preserving Dynamic Attention for Long-Term Spatial-Temporal Prediction
• Authors: Haoxing Lin, Rufan Bai, Weijia Jia, Xinyu Yang, Yongjian You
• Abstract summary: Long-term prediction is highly error-sensitive, which becomes more critical when predicting urban-wise phenomena. We propose a Dynamic Switch-Attention Network (DSAN) with a novel Multi-Space Attention (MSA) mechanism. We demonstrate the superior advantage of DSAN in both short-term and long-term predictions.
• Score: 24.752083385400343
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Effective long-term predictions have been increasingly demanded in urban-wise data mining systems. Many practical applications, such as accident prevention and resource pre-allocation, require an extended period for preparation. However, challenges come as long-term prediction is highly error-sensitive, which becomes more critical when predicting urban-wise phenomena with complicated and dynamic spatial-temporal correlation. Specifically, since the amount of valuable correlation is limited, enormous irrelevant features introduce noises that trigger increased prediction errors. Besides, after each time step, the errors can traverse through the correlations and reach the spatial-temporal positions in every future prediction, leading to significant error propagation. To address these issues, we propose a Dynamic Switch-Attention Network (DSAN) with a novel Multi-Space Attention (MSA) mechanism that measures the correlations between inputs and outputs explicitly. To filter out irrelevant noises and alleviate the error propagation, DSAN dynamically extracts valuable information by applying self-attention over the noisy input and bridges each output directly to the purified inputs via implementing a switch-attention mechanism. Through extensive experiments on two spatial-temporal prediction tasks, we demonstrate the superior advantage of DSAN in both short-term and long-term predictions.

Related papers

• Unsupervised Anomaly Prediction with N-BEATS and Graph Neural Network in Multi-variate Semiconductor Process Time Series [1.0874100424278175]
  anomaly prediction in semiconductor fabrication presents several critical challenges.<n>The complex interdependencies between variables complicate both anomaly prediction and root-cause-analysis.<n>This paper proposes two novel approaches to advance the field from anomaly detection to anomaly prediction.
  arXiv  Detail & Related papers  (2025-10-23T16:33:52Z)
• SynCast: Synergizing Contradictions in Precipitation Nowcasting via Diffusion Sequential Preference Optimization [62.958457694151384]
  We introduce preference optimization into precipitation nowcasting for the first time, motivated by the success of reinforcement learning from human feedback in large language models.<n>In the first stage, the framework focuses on reducing FAR, training the model to effectively suppress false alarms.
  arXiv  Detail & Related papers  (2025-10-22T16:11:22Z)
• RainDiff: End-to-end Precipitation Nowcasting Via Token-wise Attention Diffusion [64.49056527678606]
  We propose a Token-wise Attention integrated into not only the U-Net diffusion model but also the radar-temporal encoder.<n>Unlike prior approaches, our method integrates attention into the architecture without incurring the high resource cost typical of pixel-space diffusion.<n>Our experiments and evaluations demonstrate that the proposed method significantly outperforms state-of-the-art approaches, robustness local fidelity, generalization, and superior in complex precipitation forecasting scenarios.
  arXiv  Detail & Related papers  (2025-10-16T17:59:13Z)
• BiDepth Multimodal Neural Network: Bidirectional Depth Deep Learning Architecture for Spatial-Temporal Prediction [4.263291797886899]
  This paper proposes the BiDepth Multimodal Neural Network (BDMNN) with bidirectional depth modulation. Case studies show significant improvements in prediction accuracy, with a 12% reduction in Mean Squared Error for urban traffic prediction and a 15% improvement in rain precipitation forecasting.
  arXiv  Detail & Related papers  (2025-01-14T19:59:59Z)
• STAA: Spatio-Temporal Alignment Attention for Short-Term Precipitation Forecasting [9.177158814568887]
  Short-term precipitation forecasting model based on ontemporal alignment, with SATA as the temporal alignment module, STAU as the temporal alignment feature extractor. Based on satellite and ERA5 data, our model achieves improvements of 12.61% in terms of RMSE, in comparison with the state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-09-06T10:28:52Z)
• SFANet: Spatial-Frequency Attention Network for Weather Forecasting [54.470205739015434]
  Weather forecasting plays a critical role in various sectors, driving decision-making and risk management. Traditional methods often struggle to capture the complex dynamics of meteorological systems. We propose a novel framework designed to address these challenges and enhance the accuracy of weather prediction.
  arXiv  Detail & Related papers  (2024-05-29T08:00:15Z)
• Performative Time-Series Forecasting [71.18553214204978]
  We formalize performative time-series forecasting (PeTS) from a machine-learning perspective. We propose a novel approach, Feature Performative-Shifting (FPS), which leverages the concept of delayed response to anticipate distribution shifts. We conduct comprehensive experiments using multiple time-series models on COVID-19 and traffic forecasting tasks.
  arXiv  Detail & Related papers  (2023-10-09T18:34:29Z)
• Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglement [51.55157852647306]
  Time series forecasting has been a widely explored task of great importance in many applications. It is common that real-world time series data are recorded in a short time period, which results in a big gap between the deep model and the limited and noisy time series. We propose to address the time series forecasting problem with generative modeling and propose a bidirectional variational auto-encoder equipped with diffusion, denoise, and disentanglement.
  arXiv  Detail & Related papers  (2023-01-08T12:20:46Z)
• STG-GAN: A spatiotemporal graph generative adversarial networks for short-term passenger flow prediction in urban rail transit systems [11.167132464665578]
  Short-term passenger flow prediction is an important but challenging task for better managing urban rail transit systems. We propose a novel deep learning-basedtemporal graph generative adversarial network (STG-GAN) model with higher prediction accuracy, higher efficiency, and lower memory occupancy. This study can provide critical experience in conducting short-term passenger flow predictions, especially from the perspective of real-world applications.
  arXiv  Detail & Related papers  (2022-02-10T13:18:11Z)
• Building Autocorrelation-Aware Representations for Fine-Scale Spatiotemporal Prediction [1.2862507359003323]
  We present a novel deep learning architecture that incorporates theories of spatial statistics into neural networks. DeepLATTE contains an autocorrelation-guided semi-supervised learning strategy to enforce both local autocorrelation patterns and global autocorrelation trends. We conduct a demonstration of DeepLATTE using publicly available data for an important public health topic, air quality prediction in a well-fitting, complex physical environment.
  arXiv  Detail & Related papers  (2021-12-10T03:21:19Z)
• Multi-axis Attentive Prediction for Sparse EventData: An Application to Crime Prediction [16.654369376687296]
  We present a purely attentional approach to extract both short-term dynamics and long-term semantics of event propagation through two observation angles. The proposed contrastive learning objective significantly enhances the MAPSED's ability to capture semantics and dynamics of events.
  arXiv  Detail & Related papers  (2021-10-05T02:38:46Z)
• Adversarial Refinement Network for Human Motion Prediction [61.50462663314644]
  Two popular methods, recurrent neural networks and feed-forward deep networks, are able to predict rough motion trend. We propose an Adversarial Refinement Network (ARNet) following a simple yet effective coarse-to-fine mechanism with novel adversarial error augmentation.
  arXiv  Detail & Related papers  (2020-11-23T05:42:20Z)
• Long-Short Term Spatiotemporal Tensor Prediction for Passenger Flow Profile [15.875569404476495]
  We focus on a tensor-based prediction and propose several practical techniques to improve prediction. For long-term prediction specifically, we propose the "Tensor Decomposition + 2-Dimensional Auto-Regressive Moving Average (2D-ARMA)" model. For short-term prediction, we propose to conduct tensor completion based on tensor clustering to avoid oversimplifying and ensure accuracy.
  arXiv  Detail & Related papers  (2020-04-23T08:30:00Z)
• A Spatial-Temporal Attentive Network with Spatial Continuity for Trajectory Prediction [74.00750936752418]
  We propose a novel model named spatial-temporal attentive network with spatial continuity (STAN-SC) First, spatial-temporal attention mechanism is presented to explore the most useful and important information. Second, we conduct a joint feature sequence based on the sequence and instant state information to make the generative trajectories keep spatial continuity.
  arXiv  Detail & Related papers  (2020-03-13T04:35:50Z)
• Ambiguity in Sequential Data: Predicting Uncertain Futures with Recurrent Models [110.82452096672182]
  We propose an extension of the Multiple Hypothesis Prediction (MHP) model to handle ambiguous predictions with sequential data. We also introduce a novel metric for ambiguous problems, which is better suited to account for uncertainties.
  arXiv  Detail & Related papers  (2020-03-10T09:15:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.