Missing At Random as Covariate Shift: Correcting Bias in Iterative Imputation

• URL: http://arxiv.org/abs/2602.06713v1
• Date: Fri, 06 Feb 2026 14:02:12 GMT
• Title: Missing At Random as Covariate Shift: Correcting Bias in Iterative Imputation
• Abstract summary: We formulate missing data imputation as a risk minimisation problem.<n>We derive theoretically valid importance weights that correct for the induced distributional bias.<n>We propose a novel imputation algorithm that jointly estimates both the importance weights and imputation models.
• Score: 5.51520342433892
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate imputation of missing data is critical to downstream machine learning performance. We formulate missing data imputation as a risk minimisation problem, which highlights a covariate shift between the observed and unobserved data distributions. This covariate shift induced bias is not accounted for by popular imputation methods and leads to suboptimal performance. In this paper, we derive theoretically valid importance weights that correct for the induced distributional bias. Furthermore, we propose a novel imputation algorithm that jointly estimates both the importance weights and imputation models, enabling bias correction throughout the imputation process. Empirical results across benchmark datasets show reductions in root mean squared error and Wasserstein distance of up to 7% and 20%, respectively, compared to otherwise identical unweighted methods.

Related papers

• Distributionally Faithful Imputation via Positive Semi-Definite Kernel Density Estimation [55.47411235954028]
  We recast imputation under missing completely at random as density estimation from masked observations.<n>We obtain a convex empirical risk problem with closed form marginals, solvable by a Newton interior point method.<n>The resulting PSD Impute model yields both single and multiple imputations from the same fitted density.
  arXiv  Detail & Related papers  (2026-07-08T15:51:05Z)
• A Distributionally-Robust Framework for Nuisance in Causal Effect Estimation [1.3053649021965603]
  Causal inference requires evaluating models on balanced distributions between treatment and control groups.<n>Most conventional statistical methods address this distribution shift through inverse probability weighting (IPW)<n>We decompose the error generalization to isolate these issues--propensity ambiguity and statistical instability--and address them through an adversarial loss function.
  arXiv  Detail & Related papers  (2025-05-23T10:34:28Z)
• Revisiting the Dataset Bias Problem from a Statistical Perspective [72.94990819287551]
  We study the "dataset bias" problem from a statistical standpoint. We identify the main cause of the problem as the strong correlation between a class attribute u and a non-class attribute b. We propose to mitigate dataset bias via either weighting the objective of each sample n by frac1p(u_n|b_n) or sampling that sample with a weight proportional to frac1p(u_n|b_n).
  arXiv  Detail & Related papers  (2024-02-05T22:58:06Z)
• Continuous Treatment Effects with Surrogate Outcomes [12.548638259932915]
  We study the role of surrogates in estimating continuous treatment effects. We propose a doubly robust method to efficiently incorporate surrogates in the analysis.
  arXiv  Detail & Related papers  (2024-01-31T20:50:18Z)
• Distributed Semi-Supervised Sparse Statistical Inference [6.685997976921953]
  A debiased estimator is a crucial tool in statistical inference for high-dimensional model parameters. Traditional methods require computing a debiased estimator on every machine. An efficient multi-round distributed debiased estimator, which integrates both labeled and unlabelled data, is developed.
  arXiv  Detail & Related papers  (2023-06-17T17:30:43Z)
• Kernel-Whitening: Overcome Dataset Bias with Isotropic Sentence Embedding [51.48582649050054]
  We propose a representation normalization method which aims at disentangling the correlations between features of encoded sentences. We also propose Kernel-Whitening, a Nystrom kernel approximation method to achieve more thorough debiasing on nonlinear spurious correlations. Experiments show that Kernel-Whitening significantly improves the performance of BERT on out-of-distribution datasets while maintaining in-distribution accuracy.
  arXiv  Detail & Related papers  (2022-10-14T05:56:38Z)
• Equivariance Discovery by Learned Parameter-Sharing [153.41877129746223]
  We study how to discover interpretable equivariances from data. Specifically, we formulate this discovery process as an optimization problem over a model's parameter-sharing schemes. Also, we theoretically analyze the method for Gaussian data and provide a bound on the mean squared gap between the studied discovery scheme and the oracle scheme.
  arXiv  Detail & Related papers  (2022-04-07T17:59:19Z)
• To Impute or not to Impute? -- Missing Data in Treatment Effect Estimation [84.76186111434818]
  We identify a new missingness mechanism, which we term mixed confounded missingness (MCM), where some missingness determines treatment selection and other missingness is determined by treatment selection. We show that naively imputing all data leads to poor performing treatment effects models, as the act of imputation effectively removes information necessary to provide unbiased estimates. Our solution is selective imputation, where we use insights from MCM to inform precisely which variables should be imputed and which should not.
  arXiv  Detail & Related papers  (2022-02-04T12:08:31Z)
• Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
  Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
  arXiv  Detail & Related papers  (2022-01-11T23:01:12Z)
• Imputation-Free Learning from Incomplete Observations [73.15386629370111]
  We introduce the importance of guided gradient descent (IGSGD) method to train inference from inputs containing missing values without imputation. We employ reinforcement learning (RL) to adjust the gradients used to train the models via back-propagation. Our imputation-free predictions outperform the traditional two-step imputation-based predictions using state-of-the-art imputation methods.
  arXiv  Detail & Related papers  (2021-07-05T12:44:39Z)
• Decorrelated Clustering with Data Selection Bias [55.91842043124102]
  We propose a novel Decorrelation regularized K-Means algorithm (DCKM) for clustering with data selection bias. Our DCKM algorithm achieves significant performance gains, indicating the necessity of removing unexpected feature correlations induced by selection bias.
  arXiv  Detail & Related papers  (2020-06-29T08:55:50Z)
• Matrix Completion with Quantified Uncertainty through Low Rank Gaussian Copula [30.84155327760468]
  This paper proposes a framework for missing value imputation with quantified uncertainty. The time required to fit the model scales linearly with the number of rows and the number of columns in the dataset. Empirical results show the method yields state-of-the-art imputation accuracy across a wide range of data types.
  arXiv  Detail & Related papers  (2020-06-18T19:51:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.