Self-attention-based Diffusion Model for Time-series Imputation in Partial Blackout Scenarios

• URL: http://arxiv.org/abs/2503.01737v1
• Date: Mon, 03 Mar 2025 16:58:15 GMT
• Title: Self-attention-based Diffusion Model for Time-series Imputation in Partial Blackout Scenarios
• Authors: Mohammad Rafid Ul Islam, Prasad Tadepalli, Alan Fern,
• Abstract summary: Missing values in time series data can harm machine learning performance and introduce bias.<n>Previous work has tackled the imputation of missing data in random, complete blackouts and forecasting scenarios.<n>We introduce a two-stage imputation process using self-attention and diffusion processes to model feature and temporal correlations.
• Score: 23.160007389272575
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Missing values in multivariate time series data can harm machine learning performance and introduce bias. These gaps arise from sensor malfunctions, blackouts, and human error and are typically addressed by data imputation. Previous work has tackled the imputation of missing data in random, complete blackouts and forecasting scenarios. The current paper addresses a more general missing pattern, which we call "partial blackout," where a subset of features is missing for consecutive time steps. We introduce a two-stage imputation process using self-attention and diffusion processes to model feature and temporal correlations. Notably, our model effectively handles missing data during training, enhancing adaptability and ensuring reliable imputation and performance, even with incomplete datasets. Our experiments on benchmark and two real-world time series datasets demonstrate that our model outperforms the state-of-the-art in partial blackout scenarios and shows better scalability.

Related papers

• Retrieval-Augmented Diffusion Models for Time Series Forecasting [19.251274915003265]
  We propose a Retrieval- Augmented Time series Diffusion model (RATD) RATD consists of two parts: an embedding-based retrieval process and a reference-guided diffusion model. Our approach allows leveraging meaningful samples within the database to aid in sampling, thus maximizing the utilization of datasets.
  arXiv  Detail & Related papers  (2024-10-24T13:14:39Z)
• Test-Time Adaptation for Combating Missing Modalities in Egocentric Videos [92.38662956154256]
  Real-world applications often face challenges with incomplete modalities due to privacy concerns, efficiency needs, or hardware issues. We propose a novel approach to address this issue at test time without requiring retraining. MiDl represents the first self-supervised, online solution for handling missing modalities exclusively at test time.
  arXiv  Detail & Related papers  (2024-04-23T16:01:33Z)
• A Temporally Disentangled Contrastive Diffusion Model for Spatiotemporal Imputation [35.46631415365955]
  We introduce a conditional diffusion framework called C$2$TSD, which incorporates disentangled temporal (trend and seasonality) representations as conditional information. Our experiments on three real-world datasets demonstrate the superior performance of our approach compared to a number of state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-02-18T11:59:04Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• Deep Ensembles Meets Quantile Regression: Uncertainty-aware Imputation for Time Series [45.76310830281876]
  We propose Quantile Sub-Ensembles, a novel method to estimate uncertainty with ensemble of quantile-regression-based task networks. Our method not only produces accurate imputations that is robust to high missing rates, but also is computationally efficient due to the fast training of its non-generative model.
  arXiv  Detail & Related papers  (2023-12-03T05:52:30Z)
• An End-to-End Time Series Model for Simultaneous Imputation and Forecast [14.756607742477252]
  We develop an end-to-end time series model that aims to learn the inference relation and make a multiple-step ahead forecast. Our framework trains jointly two neural networks, one to learn the feature-wise correlations and the other for the modeling of temporal behaviors.
  arXiv  Detail & Related papers  (2023-06-01T15:08:22Z)
• DynImp: Dynamic Imputation for Wearable Sensing Data Through Sensory and Temporal Relatedness [78.98998551326812]
  We argue that traditional methods have rarely made use of both times-series dynamics of the data as well as the relatedness of the features from different sensors. We propose a model, termed as DynImp, to handle different time point's missingness with nearest neighbors along feature axis. We show that the method can exploit the multi-modality features from related sensors and also learn from history time-series dynamics to reconstruct the data under extreme missingness.
  arXiv  Detail & Related papers  (2022-09-26T21:59:14Z)
• STING: Self-attention based Time-series Imputation Networks using GAN [4.052758394413726]
  STING (Self-attention based Time-series Imputation Networks using GAN) is proposed. We take advantage of generative adversarial networks and bidirectional recurrent neural networks to learn latent representations of the time series. Experimental results on three real-world datasets demonstrate that STING outperforms the existing state-of-the-art methods in terms of imputation accuracy.
  arXiv  Detail & Related papers  (2022-09-22T06:06:56Z)
• Multi-scale Attention Flow for Probabilistic Time Series Forecasting [68.20798558048678]
  We propose a novel non-autoregressive deep learning model, called Multi-scale Attention Normalizing Flow(MANF) Our model avoids the influence of cumulative error and does not increase the time complexity. Our model achieves state-of-the-art performance on many popular multivariate datasets.
  arXiv  Detail & Related papers  (2022-05-16T07:53:42Z)
• Monte Carlo EM for Deep Time Series Anomaly Detection [6.312089019297173]
  Time series data are often corrupted by outliers or other kinds of anomalies. Recent approaches to anomaly detection and forecasting assume that the proportion of anomalies in the training data is small enough to ignore. We present a technique for augmenting existing time series models so that they explicitly account for anomalies in the training data.
  arXiv  Detail & Related papers  (2021-12-29T07:52:36Z)
• Imputation-Free Learning from Incomplete Observations [73.15386629370111]
  We introduce the importance of guided gradient descent (IGSGD) method to train inference from inputs containing missing values without imputation. We employ reinforcement learning (RL) to adjust the gradients used to train the models via back-propagation. Our imputation-free predictions outperform the traditional two-step imputation-based predictions using state-of-the-art imputation methods.
  arXiv  Detail & Related papers  (2021-07-05T12:44:39Z)
• Time-Series Imputation with Wasserstein Interpolation for Optimal Look-Ahead-Bias and Variance Tradeoff [66.59869239999459]
  In finance, imputation of missing returns may be applied prior to training a portfolio optimization model. There is an inherent trade-off between the look-ahead-bias of using the full data set for imputation and the larger variance in the imputation from using only the training data. We propose a Bayesian posterior consensus distribution which optimally controls the variance and look-ahead-bias trade-off in the imputation.
  arXiv  Detail & Related papers  (2021-02-25T09:05:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.