Branch and Bound to Assess Stability of Regression Coefficients in Uncertain Models

• URL: http://arxiv.org/abs/2408.09634v1
• Date: Mon, 19 Aug 2024 01:37:14 GMT
• Title: Branch and Bound to Assess Stability of Regression Coefficients in Uncertain Models
• Authors: Brian Knaeble, R. Mitchell Hughes, George Rudolph, Mark A. Abramson, Daniel Razo,
• Abstract summary: We introduce our algorithm, along with supporting mathematical results, an example application, and a link to our computer code. It helps researchers summarize high-dimensional data and assess the stability of regression coefficients in uncertain models.
• Score: 0.6990493129893112
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: It can be difficult to interpret a coefficient of an uncertain model. A slope coefficient of a regression model may change as covariates are added or removed from the model. In the context of high-dimensional data, there are too many model extensions to check. However, as we show here, it is possible to efficiently search, with a branch and bound algorithm, for maximum and minimum values of that adjusted slope coefficient over a discrete space of regularized regression models. Here we introduce our algorithm, along with supporting mathematical results, an example application, and a link to our computer code, to help researchers summarize high-dimensional data and assess the stability of regression coefficients in uncertain models.

Related papers

• Stochastic Gradient Descent for Nonparametric Regression [11.24895028006405]
  This paper introduces an iterative algorithm for training nonparametric additive models. We show that the resulting inequality satisfies an oracle that allows for model mis-specification.
  arXiv  Detail & Related papers  (2024-01-01T08:03:52Z)
• Probabilistic Unrolling: Scalable, Inverse-Free Maximum Likelihood Estimation for Latent Gaussian Models [69.22568644711113]
  We introduce probabilistic unrolling, a method that combines Monte Carlo sampling with iterative linear solvers to circumvent matrix inversions. Our theoretical analyses reveal that unrolling and backpropagation through the iterations of the solver can accelerate gradient estimation for maximum likelihood estimation. In experiments on simulated and real data, we demonstrate that probabilistic unrolling learns latent Gaussian models up to an order of magnitude faster than gradient EM, with minimal losses in model performance.
  arXiv  Detail & Related papers  (2023-06-05T21:08:34Z)
• Adaptive LASSO estimation for functional hidden dynamic geostatistical model [69.10717733870575]
  We propose a novel model selection algorithm based on a penalized maximum likelihood estimator (PMLE) for functional hiddenstatistical models (f-HD) The algorithm is based on iterative optimisation and uses an adaptive least absolute shrinkage and selector operator (GMSOLAS) penalty function, wherein the weights are obtained by the unpenalised f-HD maximum-likelihood estimators.
  arXiv  Detail & Related papers  (2022-08-10T19:17:45Z)
• Scalable mixed-domain Gaussian process modeling and model reduction for longitudinal data [5.00301731167245]
  We derive a basis function approximation scheme for mixed-domain covariance functions. We show that we can approximate the exact GP model accurately in a fraction of the runtime. We also demonstrate a scalable model reduction workflow for obtaining smaller and more interpretable models.
  arXiv  Detail & Related papers  (2021-11-03T04:47:37Z)
• T-LoHo: A Bayesian Regularization Model for Structured Sparsity and Smoothness on Graphs [0.0]
  In graph-structured data, structured sparsity and smoothness tend to cluster together. We propose a new prior for high dimensional parameters with graphical relations. We use it to detect structured sparsity and smoothness simultaneously.
  arXiv  Detail & Related papers  (2021-07-06T10:10:03Z)
• A Hypergradient Approach to Robust Regression without Correspondence [85.49775273716503]
  We consider a variant of regression problem, where the correspondence between input and output data is not available. Most existing methods are only applicable when the sample size is small. We propose a new computational framework -- ROBOT -- for the shuffled regression problem.
  arXiv  Detail & Related papers  (2020-11-30T21:47:38Z)
• Autoregressive Score Matching [113.4502004812927]
  We propose autoregressive conditional score models (AR-CSM) where we parameterize the joint distribution in terms of the derivatives of univariable log-conditionals (scores) For AR-CSM models, this divergence between data and model distributions can be computed and optimized efficiently, requiring no expensive sampling or adversarial training. We show with extensive experimental results that it can be applied to density estimation on synthetic data, image generation, image denoising, and training latent variable models with implicit encoders.
  arXiv  Detail & Related papers  (2020-10-24T07:01:24Z)
• Ridge Regression Revisited: Debiasing, Thresholding and Bootstrap [4.142720557665472]
  ridge regression may be worth another look since -- after debiasing and thresholding -- it may offer some advantages over the Lasso. In this paper, we define a debiased and thresholded ridge regression method, and prove a consistency result and a Gaussian approximation theorem. In addition to estimation, we consider the problem of prediction, and present a novel, hybrid bootstrap algorithm tailored for prediction intervals.
  arXiv  Detail & Related papers  (2020-09-17T05:04:10Z)
• Bayesian Sparse Covariance Structure Analysis for Correlated Count Data [3.867363075280544]
  We assume a Gaussian Graphical Model for the latent variables which dominate the potential risks of crimes. We apply the proposed model for estimation of the sparse inverse covariance of the latent variable and evaluate the partial correlation coefficients.
  arXiv  Detail & Related papers  (2020-06-05T05:34:35Z)
• Instability, Computational Efficiency and Statistical Accuracy [101.32305022521024]
  We develop a framework that yields statistical accuracy based on interplay between the deterministic convergence rate of the algorithm at the population level, and its degree of (instability) when applied to an empirical object based on $n$ samples. We provide applications of our general results to several concrete classes of models, including Gaussian mixture estimation, non-linear regression models, and informative non-response models.
  arXiv  Detail & Related papers  (2020-05-22T22:30:52Z)
• SUMO: Unbiased Estimation of Log Marginal Probability for Latent Variable Models [80.22609163316459]
  We introduce an unbiased estimator of the log marginal likelihood and its gradients for latent variable models based on randomized truncation of infinite series. We show that models trained using our estimator give better test-set likelihoods than a standard importance-sampling based approach for the same average computational cost.
  arXiv  Detail & Related papers  (2020-04-01T11:49:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.