Robust Variable Selection and Estimation Via Adaptive Elastic Net S-Estimators for Linear Regression

• URL: http://arxiv.org/abs/2107.03325v1
• Date: Wed, 7 Jul 2021 16:04:08 GMT
• Title: Robust Variable Selection and Estimation Via Adaptive Elastic Net S-Estimators for Linear Regression
• Authors: David Kepplinger
• Abstract summary: We propose a new robust regularized estimator for simultaneous variable selection and coefficient estimation. adaptive PENSE possesses the oracle property without prior knowledge of the scale of the residuals. Numerical studies on simulated and real data sets highlight superior finite-sample performance in a vast range of settings.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Heavy-tailed error distributions and predictors with anomalous values are ubiquitous in high-dimensional regression problems and can seriously jeopardize the validity of statistical analyses if not properly addressed. For more reliable estimation under these adverse conditions, we propose a new robust regularized estimator for simultaneous variable selection and coefficient estimation. This estimator, called adaptive PENSE, possesses the oracle property without prior knowledge of the scale of the residuals and without any moment conditions on the error distribution. The proposed estimator gives reliable results even under very heavy-tailed error distributions and aberrant contamination in the predictors or residuals. Importantly, even in these challenging settings variable selection by adaptive PENSE remains stable. Numerical studies on simulated and real data sets highlight superior finite-sample performance in a vast range of settings compared to other robust regularized estimators in the case of contaminated samples and competitiveness compared to classical regularized estimators in clean samples.

Related papers

• Evidential time-to-event prediction model with well-calibrated uncertainty estimation [12.446406577462069]
  We introduce an evidential regression model designed especially for time-to-event prediction tasks. The most plausible event time is directly quantified by aggregated Gaussian random fuzzy numbers (GRFNs) Our model achieves both accurate and reliable performance, outperforming state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-11-12T15:06:04Z)
• Distributionally Robust Instrumental Variables Estimation [10.765695227417865]
  We propose a distributionally robust framework for instrumental variables (IV) estimation. We show that Wasserstein DRIVE could be preferable in practice, particularly when the practitioner is uncertain about model assumptions or distributional shifts in data.
  arXiv  Detail & Related papers  (2024-10-21T04:33:38Z)
• Predictability Analysis of Regression Problems via Conditional Entropy Estimations [1.8913544072080544]
  Conditional entropy estimators are developed to assess predictability in regression problems. Experiments on synthesized and real-world datasets demonstrate the robustness and utility of these estimators.
  arXiv  Detail & Related papers  (2024-06-06T07:59:19Z)
• Relaxed Quantile Regression: Prediction Intervals for Asymmetric Noise [51.87307904567702]
  Quantile regression is a leading approach for obtaining such intervals via the empirical estimation of quantiles in the distribution of outputs. We propose Relaxed Quantile Regression (RQR), a direct alternative to quantile regression based interval construction that removes this arbitrary constraint. We demonstrate that this added flexibility results in intervals with an improvement in desirable qualities.
  arXiv  Detail & Related papers  (2024-06-05T13:36:38Z)
• Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting [55.17761802332469]
  Test-time adaptation (TTA) seeks to tackle potential distribution shifts between training and test data by adapting a given model w.r.t. any test sample. Prior methods perform backpropagation for each test sample, resulting in unbearable optimization costs to many applications. We propose an Efficient Anti-Forgetting Test-Time Adaptation (EATA) method which develops an active sample selection criterion to identify reliable and non-redundant samples.
  arXiv  Detail & Related papers  (2024-03-18T05:49:45Z)
• Selective Nonparametric Regression via Testing [54.20569354303575]
  We develop an abstention procedure via testing the hypothesis on the value of the conditional variance at a given point. Unlike existing methods, the proposed one allows to account not only for the value of the variance itself but also for the uncertainty of the corresponding variance predictor.
  arXiv  Detail & Related papers  (2023-09-28T13:04:11Z)
• The Adaptive $τ$-Lasso: Robustness and Oracle Properties [12.06248959194646]
  This paper introduces a new regularized version of the robust $tau$-regression estimator for analyzing high-dimensional datasets. The resulting estimator, termed adaptive $tau$-Lasso, is robust to outliers and high-leverage points. In the face of outliers and high-leverage points, the adaptive $tau$-Lasso and $tau$-Lasso estimators achieve the best performance or close-to-best performance.
  arXiv  Detail & Related papers  (2023-04-18T21:34:14Z)
• Improving Adaptive Conformal Prediction Using Self-Supervised Learning [72.2614468437919]
  We train an auxiliary model with a self-supervised pretext task on top of an existing predictive model and use the self-supervised error as an additional feature to estimate nonconformity scores. We empirically demonstrate the benefit of the additional information using both synthetic and real data on the efficiency (width), deficit, and excess of conformal prediction intervals.
  arXiv  Detail & Related papers  (2023-02-23T18:57:14Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)
• Distributional robustness of K-class estimators and the PULSE [4.56877715768796]
  We prove that the classical K-class estimator satisfies such optimality by establishing a connection between K-class estimators and anchor regression. We show that it can be computed efficiently as a data-driven simulation K-class estimator. There are several settings including weak instrument settings, where it outperforms other estimators.
  arXiv  Detail & Related papers  (2020-05-07T09:39:07Z)
• Estimating Gradients for Discrete Random Variables by Sampling without Replacement [93.09326095997336]
  We derive an unbiased estimator for expectations over discrete random variables based on sampling without replacement. We show that our estimator can be derived as the Rao-Blackwellization of three different estimators.
  arXiv  Detail & Related papers  (2020-02-14T14:15:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.