DiffImp: Efficient Diffusion Model for Probabilistic Time Series Imputation with Bidirectional Mamba Backbone

• URL: http://arxiv.org/abs/2410.13338v1
• Date: Thu, 17 Oct 2024 08:48:52 GMT
• Title: DiffImp: Efficient Diffusion Model for Probabilistic Time Series Imputation with Bidirectional Mamba Backbone
• Authors: Hongfan Gao, Wangmeng Shen, Xiangfei Qiu, Ronghui Xu, Jilin Hu, Bin Yang,
• Abstract summary: Current DDPM-based probabilistic time series imputation methodologies are confronted with two types of challenges. We integrate the computational efficient state space model, namely Mamba, as the backbone denosing module for DDPMs. Our approach can achieve state-of-the-art time series imputation results on multiple datasets, different missing scenarios and missing ratios.
• Score: 6.428451261614519
• License:
• Abstract: Probabilistic time series imputation has been widely applied in real-world scenarios due to its ability to estimate uncertainty of imputation results. Meanwhile, denoising diffusion probabilistic models (DDPMs) have achieved great success in probabilistic time series imputation tasks with its power to model complex distributions. However, current DDPM-based probabilistic time series imputation methodologies are confronted with two types of challenges: 1)~\textit{~The backbone modules of the denoising parts are not capable of achieving sequence modeling with low time complexity.} 2)~\textit{The architecture of denoising modules can not handle the inter-variable and bidirectional dependencies in the time series imputation problem effectively.} To address the first challenge, we integrate the computational efficient state space model, namely Mamba, as the backbone denosing module for DDPMs. To tackle the second challenge, we carefully devise several SSM-based blocks for bidirectional modeling and inter-variable relation understanding. Experimental results demonstrate that our approach can achieve state-of-the-art time series imputation results on multiple datasets, different missing scenarios and missing ratios.

Related papers

• Adaptive Multi-Scale Decomposition Framework for Time Series Forecasting [26.141054975797868]
  We propose a novel Adaptive Multi-Scale Decomposition (AMD) framework for time series forecasting (TSF) Our framework decomposes time series into distinct temporal patterns at multiple scales, leveraging the Multi-Scale Decomposable Mixing (MDM) block. Our approach effectively models both temporal and channel dependencies and utilizes autocorrelation to refine multi-scale data integration.
  arXiv  Detail & Related papers  (2024-06-06T05:27:33Z)
• SPMamba: State-space model is all you need in speech separation [20.168153319805665]
  CNN-based speech separation models face local receptive field limitations and cannot effectively capture long time dependencies. We introduce an innovative speech separation method called SPMamba. This model builds upon the robust TF-GridNet architecture, replacing its traditional BLSTM modules with bidirectional Mamba modules.
  arXiv  Detail & Related papers  (2024-04-02T16:04:31Z)
• Revitalizing Multivariate Time Series Forecasting: Learnable Decomposition with Inter-Series Dependencies and Intra-Series Variations Modeling [14.170879566023098]
  We introduce a learnable decomposition strategy to capture dynamic trend information more reasonably. We also propose a dual attention module tailored to capture inter-series dependencies and intra-series variations simultaneously.
  arXiv  Detail & Related papers  (2024-02-20T03:45:59Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• DiffSTG: Probabilistic Spatio-Temporal Graph Forecasting with Denoising Diffusion Models [53.67562579184457]
  This paper focuses on probabilistic STG forecasting, which is challenging due to the difficulty in modeling uncertainties and complex dependencies. We present the first attempt to generalize the popular denoising diffusion models to STGs, leading to a novel non-autoregressive framework called DiffSTG. Our approach combines the intrinsic-temporal learning capabilities STNNs with the uncertainty measurements of diffusion models.
  arXiv  Detail & Related papers  (2023-01-31T13:42:36Z)
• Multi-scale Attention Flow for Probabilistic Time Series Forecasting [68.20798558048678]
  We propose a novel non-autoregressive deep learning model, called Multi-scale Attention Normalizing Flow(MANF) Our model avoids the influence of cumulative error and does not increase the time complexity. Our model achieves state-of-the-art performance on many popular multivariate datasets.
  arXiv  Detail & Related papers  (2022-05-16T07:53:42Z)
• SAITS: Self-Attention-based Imputation for Time Series [6.321652307514677]
  SAITS is a novel method based on the self-attention mechanism for missing value imputation in time series. It learns missing values from a weighted combination of two diagonally-masked self-attention blocks. Tests show SAITS outperforms state-of-the-art methods on the time-series imputation task efficiently.
  arXiv  Detail & Related papers  (2022-02-17T08:40:42Z)
• CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation [107.63407690972139]
  Conditional Score-based Diffusion models for Imputation (CSDI) is a novel time series imputation method that utilizes score-based diffusion models conditioned on observed data. CSDI improves by 40-70% over existing probabilistic imputation methods on popular performance metrics. In addition, C reduces the error by 5-20% compared to the state-of-the-art deterministic imputation methods.
  arXiv  Detail & Related papers  (2021-07-07T22:20:24Z)
• NRTSI: Non-Recurrent Time Series Imputation for Irregularly-sampled Data [14.343059464246425]
  Time series imputation is a fundamental task for understanding time series with missing data. We propose a novel imputation model called NRTSI without any recurrent modules. NRTSI can easily handle irregularly-sampled data, perform multiple-mode imputation, and handle the scenario where dimensions are partially observed.
  arXiv  Detail & Related papers  (2021-02-05T18:41:25Z)
• Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encoder [59.69303945834122]
  We present a Smoothness-Inducing Sequential Variational Auto-Encoder (SISVAE) model for robust estimation and anomaly detection of time series. Our model parameterizes mean and variance for each time-stamp with flexible neural networks. We show the effectiveness of our model on both synthetic datasets and public real-world benchmarks.
  arXiv  Detail & Related papers  (2021-02-02T06:15:15Z)
• Convolutional Tensor-Train LSTM for Spatio-temporal Learning [116.24172387469994]
  We propose a higher-order LSTM model that can efficiently learn long-term correlations in the video sequence. This is accomplished through a novel tensor train module that performs prediction by combining convolutional features across time. Our results achieve state-of-the-art performance-art in a wide range of applications and datasets.
  arXiv  Detail & Related papers  (2020-02-21T05:00:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.