When Pattern-by-Pattern Works: Theoretical and Empirical Insights for Logistic Models with Missing Values

• URL: http://arxiv.org/abs/2507.13024v1
• Date: Thu, 17 Jul 2025 11:52:27 GMT
• Title: When Pattern-by-Pattern Works: Theoretical and Empirical Insights for Logistic Models with Missing Values
• Authors: Christophe Muller, Erwan Scornet, Julie Josse,
• Abstract summary: We prove that a Pattern-by-Pattern strategy (PbP) accurately approximates Bayes probabilities in various missing data scenarios.<n>Our analysis provides a comprehensive view on the logistic regression with missing values.
• Score: 10.051332392614368
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Predicting a response with partially missing inputs remains a challenging task even in parametric models, since parameter estimation in itself is not sufficient to predict on partially observed inputs. Several works study prediction in linear models. In this paper, we focus on logistic models, which present their own difficulties. From a theoretical perspective, we prove that a Pattern-by-Pattern strategy (PbP), which learns one logistic model per missingness pattern, accurately approximates Bayes probabilities in various missing data scenarios (MCAR, MAR and MNAR). Empirically, we thoroughly compare various methods (constant and iterative imputations, complete case analysis, PbP, and an EM algorithm) across classification, probability estimation, calibration, and parameter inference. Our analysis provides a comprehensive view on the logistic regression with missing values. It reveals that mean imputation can be used as baseline for low sample sizes, and improved performance is obtained via nonlinear multiple iterative imputation techniques with the labels (MICE.RF.Y). For large sample sizes, PbP is the best method for Gaussian mixtures, and we recommend MICE.RF.Y in presence of nonlinear features.

Related papers

• Recursive Equations For Imputation Of Missing Not At Random Data With Sparse Pattern Support [8.863778901027061]
  A common approach for handling missing values in data analysis pipelines is multiple imputation via software packages.<n>We develop a new characterization for the full data law in graphical models of missing data.<n>We show MISPR obtains comparable results to MICE when data are MAR, and superior, less biased results when data are MNAR.
  arXiv  Detail & Related papers  (2025-07-21T23:18:36Z)
• SubSearch: Robust Estimation and Outlier Detection for Stochastic Block Models via Subgraph Search [2.082364067210557]
  We propose an algorithm for robustly estimating SBM parameters by exploring the space of subgraphs in search of one that closely aligns with the model's assumptions.<n>Our approach also functions as an outlier detection method, properly identifying nodes responsible for the graph's deviation from the model and going beyond simple techniques like pruning high-degree nodes.
  arXiv  Detail & Related papers  (2025-06-04T07:47:25Z)
• Self-Boost via Optimal Retraining: An Analysis via Approximate Message Passing [58.52119063742121]
  Retraining a model using its own predictions together with the original, potentially noisy labels is a well-known strategy for improving the model performance.<n>This paper addresses the question of how to optimally combine the model's predictions and the provided labels.<n>Our main contribution is the derivation of the Bayes optimal aggregator function to combine the current model's predictions and the given labels.
  arXiv  Detail & Related papers  (2025-05-21T07:16:44Z)
• Adaptive Nonparametric Perturbations of Parametric Bayesian Models [33.85958872117418]
  We study nonparametrically perturbed parametric (NPP) Bayesian models, in which a parametric Bayesian model is relaxed via a distortion of its likelihood.<n>We show that NPP models can offer the robustness of non models while retaining the data efficiency of parametric models.<n>We demonstrate our method by estimating causal effects of gene expression from single cell RNA sequencing data.
  arXiv  Detail & Related papers  (2024-12-14T05:06:38Z)
• Differentiable Calibration of Inexact Stochastic Simulation Models via Kernel Score Minimization [11.955062839855334]
  We propose to learn differentiable input parameters of simulation models using output-level data via kernel score minimization with gradient descent. We quantify the uncertainties of the learned input parameters using a new normality result that accounts for model inexactness.
  arXiv  Detail & Related papers  (2024-11-08T04:13:52Z)
• Learning from aggregated data with a maximum entropy model [73.63512438583375]
  We show how a new model, similar to a logistic regression, may be learned from aggregated data only by approximating the unobserved feature distribution with a maximum entropy hypothesis. We present empirical evidence on several public datasets that the model learned this way can achieve performances comparable to those of a logistic model trained with the full unaggregated data.
  arXiv  Detail & Related papers  (2022-10-05T09:17:27Z)
• Sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithm [62.997667081978825]
  We propose a computationally efficient and powerful Bayesian approach for sparse high-dimensional linear regression. Minimal prior assumptions on the parameters are used through the use of plug-in empirical Bayes estimates. The proposed approach is implemented in the R package probe.
  arXiv  Detail & Related papers  (2022-09-16T19:15:50Z)
• Inverting brain grey matter models with likelihood-free inference: a tool for trustable cytoarchitecture measurements [62.997667081978825]
  characterisation of the brain grey matter cytoarchitecture with quantitative sensitivity to soma density and volume remains an unsolved challenge in dMRI. We propose a new forward model, specifically a new system of equations, requiring a few relatively sparse b-shells. We then apply modern tools from Bayesian analysis known as likelihood-free inference (LFI) to invert our proposed model.
  arXiv  Detail & Related papers  (2021-11-15T09:08:27Z)
• Nonparametric Functional Analysis of Generalized Linear Models Under Nonlinear Constraints [0.0]
  This article introduces a novel nonparametric methodology for Generalized Linear Models. It combines the strengths of the binary regression and latent variable formulations for categorical data. It extends recently published parametric versions of the methodology and generalizes it.
  arXiv  Detail & Related papers  (2021-10-11T04:49:59Z)
• Imputation-Free Learning from Incomplete Observations [73.15386629370111]
  We introduce the importance of guided gradient descent (IGSGD) method to train inference from inputs containing missing values without imputation. We employ reinforcement learning (RL) to adjust the gradients used to train the models via back-propagation. Our imputation-free predictions outperform the traditional two-step imputation-based predictions using state-of-the-art imputation methods.
  arXiv  Detail & Related papers  (2021-07-05T12:44:39Z)
• An interpretable prediction model for longitudinal dispersion coefficient in natural streams based on evolutionary symbolic regression network [30.99493442296212]
  Various methods have been proposed for predictions of longitudinal dispersion coefficient(LDC) In this paper, we first present an in-depth analysis of those methods and find out their defects. We then design a novel symbolic regression method called evolutionary symbolic regression network(ESRN)
  arXiv  Detail & Related papers  (2021-06-17T07:06:05Z)
• Probabilistic Circuits for Variational Inference in Discrete Graphical Models [101.28528515775842]
  Inference in discrete graphical models with variational methods is difficult. Many sampling-based methods have been proposed for estimating Evidence Lower Bound (ELBO) We propose a new approach that leverages the tractability of probabilistic circuit models, such as Sum Product Networks (SPN) We show that selective-SPNs are suitable as an expressive variational distribution, and prove that when the log-density of the target model is aweighted the corresponding ELBO can be computed analytically.
  arXiv  Detail & Related papers  (2020-10-22T05:04:38Z)
• Good Classifiers are Abundant in the Interpolating Regime [64.72044662855612]
  We develop a methodology to compute precisely the full distribution of test errors among interpolating classifiers. We find that test errors tend to concentrate around a small typical value $varepsilon*$, which deviates substantially from the test error of worst-case interpolating model. Our results show that the usual style of analysis in statistical learning theory may not be fine-grained enough to capture the good generalization performance observed in practice.
  arXiv  Detail & Related papers  (2020-06-22T21:12:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.