Fast marginal likelihood estimation of penalties for group-adaptive elastic net

• URL: http://arxiv.org/abs/2101.03875v1
• Date: Mon, 11 Jan 2021 13:30:24 GMT
• Title: Fast marginal likelihood estimation of penalties for group-adaptive elastic net
• Authors: Mirrelijn M. van Nee, Tim van de Brug, Mark A. van de Wiel
• Abstract summary: Group-adaptive elastic net penalisation learns from co-data to improve prediction. We present a fast method for marginal likelihood estimation of group-adaptive elastic net penalties for generalised linear models. We demonstrate the method in a model-based simulation study and an application to cancer genomics.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Nowadays, clinical research routinely uses omics data, such as gene expression, for predicting clinical outcomes or selecting markers. Additionally, so-called co-data are often available, providing complementary information on the covariates, like p-values from previously published studies or groups of genes corresponding to pathways. Elastic net penalisation is widely used for prediction and covariate selection. Group-adaptive elastic net penalisation learns from co-data to improve the prediction and covariate selection, by penalising important groups of covariates less than other groups. Existing methods are, however, computationally expensive. Here we present a fast method for marginal likelihood estimation of group-adaptive elastic net penalties for generalised linear models. We first derive a low-dimensional representation of the Taylor approximation of the marginal likelihood and its first derivative for group-adaptive ridge penalties, to efficiently estimate these penalties. Then we show by using asymptotic normality of the linear predictors that the marginal likelihood for elastic net models may be approximated well by the marginal likelihood for ridge models. The ridge group penalties are then transformed to elastic net group penalties by using the variance function. The method allows for overlapping groups and unpenalised variables. We demonstrate the method in a model-based simulation study and an application to cancer genomics. The method substantially decreases computation time and outperforms or matches other methods by learning from co-data.

Related papers

• Nonlinear Permuted Granger Causality [0.6526824510982799]
  Granger causal inference is a contentious but widespread method used in fields ranging from economics to neuroscience. To allow for out-of-sample comparison, a measure of functional connectivity is explicitly defined using permutations of the covariate set. Performance of the permutation method is compared to penalized variable selection, naive replacement, and omission techniques via simulation.
  arXiv  Detail & Related papers  (2023-08-11T16:44:16Z)
• Variational Classification [51.2541371924591]
  We derive a variational objective to train the model, analogous to the evidence lower bound (ELBO) used to train variational auto-encoders. Treating inputs to the softmax layer as samples of a latent variable, our abstracted perspective reveals a potential inconsistency. We induce a chosen latent distribution, instead of the implicit assumption found in a standard softmax layer.
  arXiv  Detail & Related papers  (2023-05-17T17:47:19Z)
• The non-overlapping statistical approximation to overlapping group lasso [4.197110761923662]
  We propose a separable penalty as an approximation of the overlapping group lasso penalty. Thanks to the separability, the computation of regularization based on our penalty is substantially faster than that of the overlapping group lasso. We show that the estimator based on the proposed separable penalty is statistically equivalent to the one based on the overlapping group lasso penalty.
  arXiv  Detail & Related papers  (2022-11-16T21:21:41Z)
• Modeling High-Dimensional Data with Unknown Cut Points: A Fusion Penalized Logistic Threshold Regression [2.520538806201793]
  In traditional logistic regression models, the link function is often assumed to be linear and continuous in predictors. We consider a threshold model that all continuous features are discretized into ordinal levels, which further determine the binary responses. We find the lasso model is well suited in the problem of early detection and prediction for chronic disease like diabetes.
  arXiv  Detail & Related papers  (2022-02-17T04:16:40Z)
• Long Story Short: Omitted Variable Bias in Causal Machine Learning [26.60315380737132]
  We develop a theory of omitted variable bias for a wide range of common causal parameters. We show how simple plausibility judgments on the maximum explanatory power of omitted variables are sufficient to bound the magnitude of the bias. We provide flexible and efficient statistical inference methods for the bounds, which can leverage modern machine learning algorithms for estimation.
  arXiv  Detail & Related papers  (2021-12-26T15:38:23Z)
• Selective Regression Under Fairness Criteria [30.672082160544996]
  In some cases, the performance of minority group can decrease while we reduce the coverage. We show that such an unwanted behavior can be avoided if we can construct features satisfying the sufficiency criterion.
  arXiv  Detail & Related papers  (2021-10-28T19:05:12Z)
• Near-optimal inference in adaptive linear regression [60.08422051718195]
  Even simple methods like least squares can exhibit non-normal behavior when data is collected in an adaptive manner. We propose a family of online debiasing estimators to correct these distributional anomalies in at least squares estimation. We demonstrate the usefulness of our theory via applications to multi-armed bandit, autoregressive time series estimation, and active learning with exploration.
  arXiv  Detail & Related papers  (2021-07-05T21:05:11Z)
• Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning [78.83598532168256]
  Marginal-likelihood based model-selection is rarely used in deep learning due to estimation difficulties. Our work shows that marginal likelihoods can improve generalization and be useful when validation data is unavailable.
  arXiv  Detail & Related papers  (2021-04-11T09:50:24Z)
• Increasing the efficiency of randomized trial estimates via linear adjustment for a prognostic score [59.75318183140857]
  Estimating causal effects from randomized experiments is central to clinical research. Most methods for historical borrowing achieve reductions in variance by sacrificing strict type-I error rate control.
  arXiv  Detail & Related papers  (2020-12-17T21:10:10Z)
• Autoregressive Score Matching [113.4502004812927]
  We propose autoregressive conditional score models (AR-CSM) where we parameterize the joint distribution in terms of the derivatives of univariable log-conditionals (scores) For AR-CSM models, this divergence between data and model distributions can be computed and optimized efficiently, requiring no expensive sampling or adversarial training. We show with extensive experimental results that it can be applied to density estimation on synthetic data, image generation, image denoising, and training latent variable models with implicit encoders.
  arXiv  Detail & Related papers  (2020-10-24T07:01:24Z)
• Slice Sampling for General Completely Random Measures [74.24975039689893]
  We present a novel Markov chain Monte Carlo algorithm for posterior inference that adaptively sets the truncation level using auxiliary slice variables. The efficacy of the proposed algorithm is evaluated on several popular nonparametric models.
  arXiv  Detail & Related papers  (2020-06-24T17:53:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.