Missing data imputation for noisy time-series data and applications in healthcare

• URL: http://arxiv.org/abs/2412.11164v1
• Date: Sun, 15 Dec 2024 12:23:20 GMT
• Title: Missing data imputation for noisy time-series data and applications in healthcare
• Authors: Lien P. Le, Xuan-Hien Nguyen Thi, Thu Nguyen, Michael A. Riegler, Pål Halvorsen, Binh T. Nguyen,
• Abstract summary: Imputation, i.e., filling in the missing values, is a common way to deal with noisy, missing time series data. In this study, we compare imputation methods, including Multiple Imputation with Random Forest (MICE-RF) and advanced deep learning approaches. Our results show that MICE-RF can effectively impute missing data compared to deep learning methods.
• Score: 5.586166090905021
• License:
• Abstract: Healthcare time series data is vital for monitoring patient activity but often contains noise and missing values due to various reasons such as sensor errors or data interruptions. Imputation, i.e., filling in the missing values, is a common way to deal with this issue. In this study, we compare imputation methods, including Multiple Imputation with Random Forest (MICE-RF) and advanced deep learning approaches (SAITS, BRITS, Transformer) for noisy, missing time series data in terms of MAE, F1-score, AUC, and MCC, across missing data rates (10 % - 80 %). Our results show that MICE-RF can effectively impute missing data compared to deep learning methods and the improvement in classification of data imputed indicates that imputation can have denoising effects. Therefore, using an imputation algorithm on time series with missing data can, at the same time, offer denoising effects.

Related papers

• DiffImpute: Tabular Data Imputation With Denoising Diffusion Probabilistic Model [9.908561639396273]
  We propose DiffImpute, a novel Denoising Diffusion Probabilistic Model (DDPM) It produces credible imputations for missing entries without undermining the authenticity of the existing data. It can be applied to various settings of Missing Completely At Random (MCAR) and Missing At Random (MAR)
  arXiv  Detail & Related papers  (2024-03-20T08:45:31Z)
• On the Performance of Imputation Techniques for Missing Values on Healthcare Datasets [0.0]
  Missing values or data is one popular characteristic of real-world datasets, especially healthcare data. This study is to compare the performance of seven imputation techniques, namely Mean imputation, Median Imputation, Last Observation carried Forward (LOCF) imputation, K-Nearest Neighbor (KNN) imputation, Interpolation imputation, Missforest imputation, and Multiple imputation by Chained Equations (MICE) The results show that Missforest imputation performs the best followed by MICE imputation.
  arXiv  Detail & Related papers  (2024-03-13T18:07:17Z)
• Stochastic Amortization: A Unified Approach to Accelerate Feature and Data Attribution [62.71425232332837]
  We show that training amortized models with noisy labels is inexpensive and surprisingly effective. This approach significantly accelerates several feature attribution and data valuation methods, often yielding an order of magnitude speedup over existing approaches.
  arXiv  Detail & Related papers  (2024-01-29T03:42:37Z)
• Robust T-Loss for Medical Image Segmentation [56.524774292536264]
  This paper presents a new robust loss function, the T-Loss, for medical image segmentation. The proposed loss is based on the negative log-likelihood of the Student-t distribution and can effectively handle outliers in the data. Our experiments show that the T-Loss outperforms traditional loss functions in terms of dice scores on two public medical datasets.
  arXiv  Detail & Related papers  (2023-06-01T14:49:40Z)
• Filling out the missing gaps: Time Series Imputation with Semi-Supervised Learning [7.8379910349669]
  We propose a semi-supervised imputation method, ST-Impute, that uses both unlabeled data along with downstream task's labeled data. ST-Impute is based on sparse self-attention and trains on tasks that mimic the imputation process.
  arXiv  Detail & Related papers  (2023-04-09T16:38:47Z)
• A robust deep learning-based damage identification approach for SHM considering missing data [12.46223206282221]
  Missing data significantly impacts the conduction of Structural Health Monitoring method. This paper develops a robust method for damage identification that considers the missing data occasions. Results show that the missing data imputation and damage identification can be implemented together.
  arXiv  Detail & Related papers  (2023-03-31T18:00:56Z)
• Deep Imputation of Missing Values in Time Series Health Data: A Review with Benchmarking [0.0]
  This survey performs six data-centric experiments to benchmark state-of-the-art deep imputation methods on five time series health data sets. Deep learning methods that jointly perform cross-sectional (across variables) and longitudinal (across time) imputations of missing values in time series data yield statistically better data quality than traditional imputation methods.
  arXiv  Detail & Related papers  (2023-02-10T16:03:36Z)
• Improving the Robustness of Summarization Models by Detecting and Removing Input Noise [50.27105057899601]
  We present a large empirical study quantifying the sometimes severe loss in performance from different types of input noise for a range of datasets and model sizes. We propose a light-weight method for detecting and removing such noise in the input during model inference without requiring any training, auxiliary models, or even prior knowledge of the type of noise.
  arXiv  Detail & Related papers  (2022-12-20T00:33:11Z)
• DynImp: Dynamic Imputation for Wearable Sensing Data Through Sensory and Temporal Relatedness [78.98998551326812]
  We argue that traditional methods have rarely made use of both times-series dynamics of the data as well as the relatedness of the features from different sensors. We propose a model, termed as DynImp, to handle different time point's missingness with nearest neighbors along feature axis. We show that the method can exploit the multi-modality features from related sensors and also learn from history time-series dynamics to reconstruct the data under extreme missingness.
  arXiv  Detail & Related papers  (2022-09-26T21:59:14Z)
• MissDAG: Causal Discovery in the Presence of Missing Data with Continuous Additive Noise Models [78.72682320019737]
  We develop a general method, which we call MissDAG, to perform causal discovery from data with incomplete observations. MissDAG maximizes the expected likelihood of the visible part of observations under the expectation-maximization framework. We demonstrate the flexibility of MissDAG for incorporating various causal discovery algorithms and its efficacy through extensive simulations and real data experiments.
  arXiv  Detail & Related papers  (2022-05-27T09:59:46Z)
• Multi-Source Causal Inference Using Control Variates [81.57072928775509]
  We propose a general algorithm to estimate causal effects from emphmultiple data sources. We show theoretically that this reduces the variance of the ATE estimate. We apply this framework to inference from observational data under an outcome selection bias.
  arXiv  Detail & Related papers  (2021-03-30T21:20:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.