Machine Learning under Imperfect Data: Challenges and Methods

• URL: http://arxiv.org/abs/2609.13914v1
• Date: Sat, 12 Sep 2026 12:59:23 GMT
• Title: Machine Learning under Imperfect Data: Challenges and Methods
• Abstract summary: Machine-learning models are commonly developed under an assumption that training and test data are sufficiently complete, balanced, labelled, and drawn from compatible distributions.<n>In practice, one or more of these conditions is often violated. Measurements may be missing or corrupted, rare classes may be poorly represented, supervision may be weak, and the deployment environment may differ from the training environment.<n>This survey organises representative methods around these mechanisms.<n>It reviews reconstruction and generation, rebalancing and representation calibration, learning with limited supervision, adaptation across domains and modalities, and reliability under distribution change.
• Score: 12.988380776855772
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Machine-learning models are commonly developed under an assumption that training and test data are sufficiently complete, balanced, labelled, and drawn from compatible distributions. In practice, one or more of these conditions is often violated. Measurements may be missing or corrupted, rare classes may be poorly represented, supervision may be weak, and the deployment environment may differ from the training environment. These imperfections are usually treated as separate technical problems, although they alter learning through a small number of shared mechanisms: loss of information, biased empirical risk, ambiguous supervision, and unstable representations. This short survey organises representative methods around these mechanisms. It reviews reconstruction and generation, rebalancing and representation calibration, learning with limited supervision, adaptation across domains and modalities, and reliability under distribution change. The discussion highlights the limits of plausible reconstruction, benchmark-specific correction, and adaptation without trustworthy feedback. It concludes with directions for evidence-aware learning, uncertainty-preserving prediction, and evaluation that separates visual plausibility from decision utility.

Related papers

• Discovery of Hidden Miscalibration Regimes [52.452902154360565]
  Models may be systematically overconfident on some kinds of inputs and underconfident on others.<n>We define the corresponding miscalibration field and propose a diagnostic framework for estimating it.<n>Our approach learns a calibration-aware representation of the input space and estimates signed local miscalibration by kernel smoothing in the learned geometry.
  arXiv  Detail & Related papers  (2026-05-13T13:07:50Z)
• Unified Meta-Representation and Feedback Calibration for General Disturbance Estimation [15.761366602886868]
  We present a disturbance estimation framework that builds on meta-learning and feedback-calibrated online adaptation.<n>By extracting features from a finite time window of past observations, a unified representation that effectively captures general non-structural disturbances can be learned.<n>Our framework effectively estimates multiple rapidly changing disturbances, as demonstrated by quadrotor flight experiments.
  arXiv  Detail & Related papers  (2026-01-06T06:53:19Z)
• Equivariant Splitting: Self-supervised learning from incomplete data [11.641086974203924]
  We propose a new self-supervised learning strategy devised for the challenging setting where measurements are observed via a single incomplete observation model.<n>We show that the combination of self-supervised splitting losses and equivariant reconstruction networks results in the same minimizer in expectation as the one of a supervised loss.
  arXiv  Detail & Related papers  (2025-10-01T14:08:17Z)
• From Lab to Factory: Pitfalls and Guidelines for Self-/Unsupervised Defect Detection on Low-Quality Industrial Images [1.8185814461140655]
  Machine learning has the potential to replace manual handling.<n>Most methods do not handle low data quality well or exude low robustness in unfavorable, but typical real-world settings.<n>We evaluate two types of state-of-the-art models that allow us to identify and improve quality issues in production data.
  arXiv  Detail & Related papers  (2025-06-20T10:28:00Z)
• CLUE: Neural Networks Calibration via Learning Uncertainty-Error alignment [7.702016079410588]
  We introduce CLUE (Calibration via Learning Uncertainty-Error Alignment), a novel approach that aligns predicted uncertainty with observed error during training.<n>We show that CLUE achieves superior calibration quality and competitive predictive performance with respect to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-05-28T19:23:47Z)
• Rethinking Early Stopping: Refine, Then Calibrate [49.966899634962374]
  We present a novel variational formulation of the calibration-refinement decomposition.<n>We provide theoretical and empirical evidence that calibration and refinement errors are not minimized simultaneously during training.
  arXiv  Detail & Related papers  (2025-01-31T15:03:54Z)
• Revisiting Machine Unlearning with Dimensional Alignment [46.29161970063835]
  We introduce a novel evaluation metric for machine unlearning, coined dimensional alignment, which measures the alignment between the eigenspaces of the forget and retain set samples.<n>Our framework effectively eliminates information from the forget set and preserves knowledge from the retain set.
  arXiv  Detail & Related papers  (2024-07-25T02:05:15Z)
• Arbitrariness and Social Prediction: The Confounding Role of Variance in Fair Classification [31.392067805022414]
  Variance in predictions across different trained models is a significant, under-explored source of error in fair binary classification. In practice, the variance on some data examples is so large that decisions can be effectively arbitrary. We develop an ensembling algorithm that abstains from classification when a prediction would be arbitrary.
  arXiv  Detail & Related papers  (2023-01-27T06:52:04Z)
• Trust but Verify: Assigning Prediction Credibility by Counterfactual Constrained Learning [123.3472310767721]
  Prediction credibility measures are fundamental in statistics and machine learning. These measures should account for the wide variety of models used in practice. The framework developed in this work expresses the credibility as a risk-fit trade-off.
  arXiv  Detail & Related papers  (2020-11-24T19:52:38Z)
• Accurate and Robust Feature Importance Estimation under Distribution Shifts [49.58991359544005]
  PRoFILE is a novel feature importance estimation method. We show significant improvements over state-of-the-art approaches, both in terms of fidelity and robustness.
  arXiv  Detail & Related papers  (2020-09-30T05:29:01Z)
• Learning perturbation sets for robust machine learning [97.6757418136662]
  We use a conditional generator that defines the perturbation set over a constrained region of the latent space. We measure the quality of our learned perturbation sets both quantitatively and qualitatively. We leverage our learned perturbation sets to train models which are empirically and certifiably robust to adversarial image corruptions and adversarial lighting variations.
  arXiv  Detail & Related papers  (2020-07-16T16:39:54Z)
• Learning Diverse Representations for Fast Adaptation to Distribution Shift [78.83747601814669]
  We present a method for learning multiple models, incorporating an objective that pressures each to learn a distinct way to solve the task. We demonstrate our framework's ability to facilitate rapid adaptation to distribution shift.
  arXiv  Detail & Related papers  (2020-06-12T12:23:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.