SAITS: Self-Attention-based Imputation for Time Series

• URL: http://arxiv.org/abs/2202.08516v5
• Date: Wed, 5 Jul 2023 14:53:55 GMT
• Title: SAITS: Self-Attention-based Imputation for Time Series
• Authors: Wenjie Du, David Cote, Yan Liu
• Abstract summary: SAITS is a novel method based on the self-attention mechanism for missing value imputation in time series. It learns missing values from a weighted combination of two diagonally-masked self-attention blocks. Tests show SAITS outperforms state-of-the-art methods on the time-series imputation task efficiently.
• Score: 6.321652307514677
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Missing data in time series is a pervasive problem that puts obstacles in the way of advanced analysis. A popular solution is imputation, where the fundamental challenge is to determine what values should be filled in. This paper proposes SAITS, a novel method based on the self-attention mechanism for missing value imputation in multivariate time series. Trained by a joint-optimization approach, SAITS learns missing values from a weighted combination of two diagonally-masked self-attention (DMSA) blocks. DMSA explicitly captures both the temporal dependencies and feature correlations between time steps, which improves imputation accuracy and training speed. Meanwhile, the weighted-combination design enables SAITS to dynamically assign weights to the learned representations from two DMSA blocks according to the attention map and the missingness information. Extensive experiments quantitatively and qualitatively demonstrate that SAITS outperforms the state-of-the-art methods on the time-series imputation task efficiently and reveal SAITS' potential to improve the learning performance of pattern recognition models on incomplete time-series data from the real world. The code is open source on GitHub at https://github.com/WenjieDu/SAITS.

Related papers

• MTSCI: A Conditional Diffusion Model for Multivariate Time Series Consistent Imputation [41.681869408967586]
  Key research question is how to ensure imputation consistency, i.e., intra-consistency between observed and imputed values. Previous methods rely solely on the inductive bias of the imputation targets to guide the learning process.
  arXiv  Detail & Related papers  (2024-08-11T10:24:53Z)
• TSI-Bench: Benchmarking Time Series Imputation [52.27004336123575]
  TSI-Bench is a comprehensive benchmark suite for time series imputation utilizing deep learning techniques. The TSI-Bench pipeline standardizes experimental settings to enable fair evaluation of imputation algorithms. TSI-Bench innovatively provides a systematic paradigm to tailor time series forecasting algorithms for imputation purposes.
  arXiv  Detail & Related papers  (2024-06-18T16:07:33Z)
• DeformTime: Capturing Variable Dependencies with Deformable Attention for Time Series Forecasting [0.34530027457862006]
  We present DeformTime, a neural network architecture that attempts to capture correlated temporal patterns from the input space. We conduct extensive experiments on 6 MTS data sets, using previously established benchmarks as well as challenging infectious disease modelling tasks. Results demonstrate that DeformTime improves accuracy against previous competitive methods across the vast majority of MTS forecasting tasks.
  arXiv  Detail & Related papers  (2024-06-11T16:45:48Z)
• Combating Missing Modalities in Egocentric Videos at Test Time [92.38662956154256]
  Real-world applications often face challenges with incomplete modalities due to privacy concerns, efficiency needs, or hardware issues. We propose a novel approach to address this issue at test time without requiring retraining. MiDl represents the first self-supervised, online solution for handling missing modalities exclusively at test time.
  arXiv  Detail & Related papers  (2024-04-23T16:01:33Z)
• Incremental Self-training for Semi-supervised Learning [56.57057576885672]
  IST is simple yet effective and fits existing self-training-based semi-supervised learning methods. We verify the proposed IST on five datasets and two types of backbone, effectively improving the recognition accuracy and learning speed.
  arXiv  Detail & Related papers  (2024-04-14T05:02:00Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• Uncertainty-Aware Deep Attention Recurrent Neural Network for Heterogeneous Time Series Imputation [0.25112747242081457]
  Missingness is ubiquitous in multivariate time series and poses an obstacle to reliable downstream analysis. We propose DEep Attention Recurrent Imputation (Imputation), which jointly estimates missing values and their associated uncertainty. Experiments show that I surpasses the SOTA in diverse imputation tasks using real-world datasets.
  arXiv  Detail & Related papers  (2024-01-04T13:21:11Z)
• STING: Self-attention based Time-series Imputation Networks using GAN [4.052758394413726]
  STING (Self-attention based Time-series Imputation Networks using GAN) is proposed. We take advantage of generative adversarial networks and bidirectional recurrent neural networks to learn latent representations of the time series. Experimental results on three real-world datasets demonstrate that STING outperforms the existing state-of-the-art methods in terms of imputation accuracy.
  arXiv  Detail & Related papers  (2022-09-22T06:06:56Z)
• Self-Attention Neural Bag-of-Features [103.70855797025689]
  We build on the recently introduced 2D-Attention and reformulate the attention learning methodology. We propose a joint feature-temporal attention mechanism that learns a joint 2D attention mask highlighting relevant information.
  arXiv  Detail & Related papers  (2022-01-26T17:54:14Z)
• Towards Similarity-Aware Time-Series Classification [51.2400839966489]
  We study time-series classification (TSC), a fundamental task of time-series data mining. We propose Similarity-Aware Time-Series Classification (SimTSC), a framework that models similarity information with graph neural networks (GNNs)
  arXiv  Detail & Related papers  (2022-01-05T02:14:57Z)
• Temporal Dependencies in Feature Importance for Time Series Predictions [4.082348823209183]
  We propose WinIT, a framework for evaluating feature importance in time series prediction settings. We demonstrate how the solution improves the appropriate attribution of features within time steps. WinIT achieves 2.47x better performance than FIT and other feature importance methods on real-world clinical MIMIC-mortality task.
  arXiv  Detail & Related papers  (2021-07-29T20:31:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.