A cost-sensitive constrained Lasso

• URL: http://arxiv.org/abs/2401.18023v1
• Date: Wed, 31 Jan 2024 17:36:21 GMT
• Title: A cost-sensitive constrained Lasso
• Authors: Rafael Blanquero, Emilio Carrizosa, Pepa Ram\'irez-Cobo, M. Remedios Sillero-Denamiel
• Abstract summary: We propose a novel version of the Lasso in which quadratic performance constraints are added to Lasso-based objective functions. As a result, a constrained sparse regression model is defined by a nonlinear optimization problem. This cost-sensitive constrained Lasso has a direct application in heterogeneous samples where data are collected from distinct sources.
• Score: 2.8265531928694116
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The Lasso has become a benchmark data analysis procedure, and numerous variants have been proposed in the literature. Although the Lasso formulations are stated so that overall prediction error is optimized, no full control over the accuracy prediction on certain individuals of interest is allowed. In this work we propose a novel version of the Lasso in which quadratic performance constraints are added to Lasso-based objective functions, in such a way that threshold values are set to bound the prediction errors in the different groups of interest (not necessarily disjoint). As a result, a constrained sparse regression model is defined by a nonlinear optimization problem. This cost-sensitive constrained Lasso has a direct application in heterogeneous samples where data are collected from distinct sources, as it is standard in many biomedical contexts. Both theoretical properties and empirical studies concerning the new method are explored in this paper. In addition, two illustrations of the method on biomedical and sociological contexts are considered.

Related papers

• Selective Nonparametric Regression via Testing [54.20569354303575]
  We develop an abstention procedure via testing the hypothesis on the value of the conditional variance at a given point. Unlike existing methods, the proposed one allows to account not only for the value of the variance itself but also for the uncertainty of the corresponding variance predictor.
  arXiv  Detail & Related papers  (2023-09-28T13:04:11Z)
• Adaptive Lasso, Transfer Lasso, and Beyond: An Asymptotic Perspective [4.051523221722475]
  This paper presents a comprehensive exploration of the theoretical properties inherent in the Adaptive Lasso and the Transfer Lasso. The Adaptive Lasso employs regularization divided by initial estimators and is characterized by normality and variable selection consistency. The recently proposed Transfer Lasso employs regularization subtracted by initial estimators with the demonstrated capacity to curtail non-asymptotic estimation errors.
  arXiv  Detail & Related papers  (2023-08-30T08:21:46Z)
• Nonlinear Permuted Granger Causality [0.6526824510982799]
  Granger causal inference is a contentious but widespread method used in fields ranging from economics to neuroscience. To allow for out-of-sample comparison, a measure of functional connectivity is explicitly defined using permutations of the covariate set. Performance of the permutation method is compared to penalized variable selection, naive replacement, and omission techniques via simulation.
  arXiv  Detail & Related papers  (2023-08-11T16:44:16Z)
• Variational Classification [51.2541371924591]
  We derive a variational objective to train the model, analogous to the evidence lower bound (ELBO) used to train variational auto-encoders. Treating inputs to the softmax layer as samples of a latent variable, our abstracted perspective reveals a potential inconsistency. We induce a chosen latent distribution, instead of the implicit assumption found in a standard softmax layer.
  arXiv  Detail & Related papers  (2023-05-17T17:47:19Z)
• Mitigating multiple descents: A model-agnostic framework for risk monotonization [84.6382406922369]
  We develop a general framework for risk monotonization based on cross-validation. We propose two data-driven methodologies, namely zero- and one-step, that are akin to bagging and boosting.
  arXiv  Detail & Related papers  (2022-05-25T17:41:40Z)
• Sparse Feature Selection Makes Batch Reinforcement Learning More Sample Efficient [62.24615324523435]
  This paper provides a statistical analysis of high-dimensional batch Reinforcement Learning (RL) using sparse linear function approximation. When there is a large number of candidate features, our result sheds light on the fact that sparsity-aware methods can make batch RL more sample efficient.
  arXiv  Detail & Related papers  (2020-11-08T16:48:02Z)
• Accounting for Unobserved Confounding in Domain Generalization [107.0464488046289]
  This paper investigates the problem of learning robust, generalizable prediction models from a combination of datasets. Part of the challenge of learning robust models lies in the influence of unobserved confounders. We demonstrate the empirical performance of our approach on healthcare data from different modalities.
  arXiv  Detail & Related papers  (2020-07-21T08:18:06Z)
• A One-step Approach to Covariate Shift Adaptation [82.01909503235385]
  A default assumption in many machine learning scenarios is that the training and test samples are drawn from the same probability distribution. We propose a novel one-step approach that jointly learns the predictive model and the associated weights in one optimization.
  arXiv  Detail & Related papers  (2020-07-08T11:35:47Z)
• Weighted Lasso Estimates for Sparse Logistic Regression: Non-asymptotic Properties with Measurement Error [5.5233023574863624]
  Two types of weighted Lasso estimates are proposed for $ell_1$-penalized logistic regression. We show that the finite sample behavior of our proposed methods with a diverging number of predictors is illustrated by non-asymptotic oracle inequalities. We compare the performance of our methods with former weighted estimates on simulated data, then apply these methods to do real data analysis.
  arXiv  Detail & Related papers  (2020-06-11T00:58:14Z)
• Generic Error Bounds for the Generalized Lasso with Sub-Exponential Data [4.56877715768796]
  This work performs a non-asymptotic analysis of the generalized Lasso under the assumption of sub-exponential data. We show that the estimation error can be controlled by means of two complexity parameters that arise naturally from a generic-chaining-based proof strategy.
  arXiv  Detail & Related papers  (2020-04-11T10:39:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.