Evaluation of Missing Data Imputation for Time Series Without Ground Truth

• URL: http://arxiv.org/abs/2503.05775v1
• Date: Wed, 26 Feb 2025 01:02:16 GMT
• Title: Evaluation of Missing Data Imputation for Time Series Without Ground Truth
• Authors: Rania Farjallah, Bassant Selim, Brigitte Jaumard, Samr Ali, Georges Kaddoum,
• Abstract summary: This paper introduces two statistical metrics, the wasserstein distance (WD) and jensen-shannon divergence (JSD)<n>WD and JSD are effective metrics for assessing the quality of missing data imputation, particularly in scenarios where ground truth data is unavailable.
• Score: 19.73474464678241
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The challenge of handling missing data in time series is critical for maintaining the accuracy and reliability of machine learning (ML) models in applications like fifth generation mobile communication (5G) network management. Traditional methods for validating imputation rely on ground truth data, which is inherently unavailable. This paper addresses this limitation by introducing two statistical metrics, the wasserstein distance (WD) and jensen-shannon divergence (JSD), to evaluate imputation quality without requiring ground truth. These metrics assess the alignment between the distributions of imputed and original data, providing a robust method for evaluating imputation performance based on internal structure and data consistency. We apply and test these metrics across several imputation techniques. Results demonstrate that WD and JSD are effective metrics for assessing the quality of missing data imputation, particularly in scenarios where ground truth data is unavailable.

Related papers

• CoSTI: Consistency Models for (a faster) Spatio-Temporal Imputation [0.0]
  CoSTI employs Consistency Training to achieve comparable imputation quality to DDPMs while drastically reducing inference times.<n>We evaluate CoSTI across multiple datasets and missing data scenarios, demonstrating up to a 98% reduction in imputation time with performance par with diffusion-based models.
  arXiv  Detail & Related papers  (2025-01-31T18:14:28Z)
• ITI-IQA: a Toolbox for Heterogeneous Univariate and Multivariate Missing Data Imputation Quality Assessment [0.0]
  ITI-IQA is a set of utilities designed to assess the reliability of various imputation methods. The toolbox also includes a suite of diagnosing methods and graphical tools to check measurements.
  arXiv  Detail & Related papers  (2024-07-16T14:26:46Z)
• DAGnosis: Localized Identification of Data Inconsistencies using Structures [73.39285449012255]
  Identification and appropriate handling of inconsistencies in data at deployment time is crucial to reliably use machine learning models. We use directed acyclic graphs (DAGs) to encode the training set's features probability distribution and independencies as a structure. Our method, called DAGnosis, leverages these structural interactions to bring valuable and insightful data-centric conclusions.
  arXiv  Detail & Related papers  (2024-02-26T11:29:16Z)
• Weak Supervision Performance Evaluation via Partial Identification [46.73061437177238]
  Programmatic Weak Supervision (PWS) enables supervised model training without direct access to ground truth labels. We present a novel method to address this challenge by framing model evaluation as a partial identification problem. Our approach derives reliable bounds on key metrics without requiring labeled data, overcoming core limitations in current weak supervision evaluation techniques.
  arXiv  Detail & Related papers  (2023-12-07T07:15:11Z)
• A robust deep learning-based damage identification approach for SHM considering missing data [12.46223206282221]
  Missing data significantly impacts the conduction of Structural Health Monitoring method. This paper develops a robust method for damage identification that considers the missing data occasions. Results show that the missing data imputation and damage identification can be implemented together.
  arXiv  Detail & Related papers  (2023-03-31T18:00:56Z)
• Deep Imputation of Missing Values in Time Series Health Data: A Review with Benchmarking [0.0]
  This survey performs six data-centric experiments to benchmark state-of-the-art deep imputation methods on five time series health data sets. Deep learning methods that jointly perform cross-sectional (across variables) and longitudinal (across time) imputations of missing values in time series data yield statistically better data quality than traditional imputation methods.
  arXiv  Detail & Related papers  (2023-02-10T16:03:36Z)
• Conditional expectation with regularization for missing data imputation [19.254291863337347]
  Missing data frequently occurs in datasets across various domains, such as medicine, sports, and finance. We propose a new algorithm named "conditional Distribution-based Imputation of Missing Values with Regularization" (DIMV) DIMV operates by determining the conditional distribution of a feature that has missing entries, using the information from the fully observed features as a basis.
  arXiv  Detail & Related papers  (2023-02-02T06:59:15Z)
• Investigating Data Variance in Evaluations of Automatic Machine Translation Metrics [58.50754318846996]
  In this paper, we show that the performances of metrics are sensitive to data. The ranking of metrics varies when the evaluation is conducted on different datasets.
  arXiv  Detail & Related papers  (2022-03-29T18:58:28Z)
• Data-SUITE: Data-centric identification of in-distribution incongruous examples [81.21462458089142]
  Data-SUITE is a data-centric framework to identify incongruous regions of in-distribution (ID) data. We empirically validate Data-SUITE's performance and coverage guarantees.
  arXiv  Detail & Related papers  (2022-02-17T18:58:31Z)
• Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
  Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
  arXiv  Detail & Related papers  (2022-01-11T23:01:12Z)
• How Training Data Impacts Performance in Learning-based Control [67.7875109298865]
  This paper derives an analytical relationship between the density of the training data and the control performance. We formulate a quality measure for the data set, which we refer to as $rho$-gap. We show how the $rho$-gap can be applied to a feedback linearizing control law.
  arXiv  Detail & Related papers  (2020-05-25T12:13:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.