An ADMM approach for multi-response regression with overlapping groups and interaction effects

• URL: http://arxiv.org/abs/2303.11155v1
• Date: Mon, 20 Mar 2023 14:43:22 GMT
• Title: An ADMM approach for multi-response regression with overlapping groups and interaction effects
• Authors: Theophilus Quachie Asenso and Manuela Zucknick
• Abstract summary: We propose MADMMplasso, a novel regularized regression method. For parameter estimation, we develop an ADMM algorithm that allows us to implement the overlapping groups in a simple way. The results from the simulations and analysis of a pharmacogenomic screen data set show that the proposed method has an advantage in handling correlated responses and interaction effects.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: In this paper, we consider the regularized multi-response regression problem where there exists some structural relation within the responses and also between the covariates and a set of modifying variables. To handle this problem, we propose MADMMplasso, a novel regularized regression method. This method is able to find covariates and their corresponding interactions, with some joint association with multiple related responses. We allow the interaction term between covariate and modifying variable to be included in a (weak) asymmetrical hierarchical manner by first considering whether the corresponding covariate main term is in the model. For parameter estimation, we develop an ADMM algorithm that allows us to implement the overlapping groups in a simple way. The results from the simulations and analysis of a pharmacogenomic screen data set show that the proposed method has an advantage in handling correlated responses and interaction effects, both with respect to prediction and variable selection performance.

Related papers

• Linked shrinkage to improve estimation of interaction effects in regression models [0.0]
  We develop an estimator that adapts well to two-way interaction terms in a regression model. We evaluate the potential of the model for inference, which is notoriously hard for selection strategies. Our models can be very competitive to a more advanced machine learner, like random forest, even for fairly large sample sizes.
  arXiv  Detail & Related papers  (2023-09-25T10:03:39Z)
• Heterogeneous Multi-Task Gaussian Cox Processes [61.67344039414193]
  We present a novel extension of multi-task Gaussian Cox processes for modeling heterogeneous correlated tasks jointly. A MOGP prior over the parameters of the dedicated likelihoods for classification, regression and point process tasks can facilitate sharing of information between heterogeneous tasks. We derive a mean-field approximation to realize closed-form iterative updates for estimating model parameters.
  arXiv  Detail & Related papers  (2023-08-29T15:01:01Z)
• HiPerformer: Hierarchically Permutation-Equivariant Transformer for Time Series Forecasting [56.95572957863576]
  We propose a hierarchically permutation-equivariant model that considers both the relationship among components in the same group and the relationship among groups. The experiments conducted on real-world data demonstrate that the proposed method outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-14T05:11:52Z)
• Covariance regression with random forests [0.0]
  CovRegRF is implemented in a freely available R package on CRAN. An application of the proposed method to thyroid disease data is also presented.
  arXiv  Detail & Related papers  (2022-09-16T21:21:18Z)
• Sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithm [62.997667081978825]
  We propose a computationally efficient and powerful Bayesian approach for sparse high-dimensional linear regression. Minimal prior assumptions on the parameters are used through the use of plug-in empirical Bayes estimates. The proposed approach is implemented in the R package probe.
  arXiv  Detail & Related papers  (2022-09-16T19:15:50Z)
• Machine Learning for Multi-Output Regression: When should a holistic multivariate approach be preferred over separate univariate ones? [62.997667081978825]
  Tree-based ensembles such as the Random Forest are modern classics among statistical learning methods. We compare these methods in extensive simulations to help in answering the primary question when to use multivariate ensemble techniques.
  arXiv  Detail & Related papers  (2022-01-14T08:44:25Z)
• CARMS: Categorical-Antithetic-REINFORCE Multi-Sample Gradient Estimator [60.799183326613395]
  We propose an unbiased estimator for categorical random variables based on multiple mutually negatively correlated (jointly antithetic) samples. CARMS combines REINFORCE with copula based sampling to avoid duplicate samples and reduce its variance, while keeping the estimator unbiased using importance sampling. We evaluate CARMS on several benchmark datasets on a generative modeling task, as well as a structured output prediction task, and find it to outperform competing methods including a strong self-control baseline.
  arXiv  Detail & Related papers  (2021-10-26T20:14:30Z)
• On the Use of Minimum Penalties in Statistical Learning [2.1320960069210475]
  We propose a framework to simultaneously estimate regression coefficients associated with a multivariate regression model and relationships between outcome variables. An iterative algorithm that generalizes current state art methods is proposed as a solution. We extend the proposed MinPen framework to other exponential family loss functions, with a specific focus on multiple binomial responses.
  arXiv  Detail & Related papers  (2021-06-09T16:15:46Z)
• A variational inference framework for inverse problems [0.39373541926236766]
  A framework is presented for fitting inverse problem models via variational Bayes approximations. This methodology guarantees flexibility to statistical model specification for a broad range of applications. An image processing application and a simulation exercise motivated by biomedical problems reveal the computational advantage offered by variational Bayes.
  arXiv  Detail & Related papers  (2021-03-10T07:37:20Z)
• The MELODIC family for simultaneous binary logistic regression in a reduced space [0.5330240017302619]
  We propose the MELODIC family for simultaneous binary logistic regression modeling. The model may be interpreted in terms of logistic regression coefficients or in terms of a biplot. Two applications are shown in detail: one relating personality characteristics to drug consumption profiles and one relating personality characteristics to depressive and anxiety disorders.
  arXiv  Detail & Related papers  (2021-02-16T15:47:20Z)
• Adaptive Correlated Monte Carlo for Contextual Categorical Sequence Generation [77.7420231319632]
  We adapt contextual generation of categorical sequences to a policy gradient estimator, which evaluates a set of correlated Monte Carlo (MC) rollouts for variance control. We also demonstrate the use of correlated MC rollouts for binary-tree softmax models, which reduce the high generation cost in large vocabulary scenarios.
  arXiv  Detail & Related papers  (2019-12-31T03:01:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.