On the Use of Minimum Penalties in Statistical Learning
- URL: http://arxiv.org/abs/2106.05172v1
- Date: Wed, 9 Jun 2021 16:15:46 GMT
- Title: On the Use of Minimum Penalties in Statistical Learning
- Authors: Ben Sherwood and Bradley S. Price
- Abstract summary: We propose a framework to simultaneously estimate regression coefficients associated with a multivariate regression model and relationships between outcome variables.
An iterative algorithm that generalizes current state art methods is proposed as a solution.
We extend the proposed MinPen framework to other exponential family loss functions, with a specific focus on multiple binomial responses.
- Score: 2.1320960069210475
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Modern multivariate machine learning and statistical methodologies estimate
parameters of interest while leveraging prior knowledge of the association
between outcome variables. The methods that do allow for estimation of
relationships do so typically through an error covariance matrix in
multivariate regression which does not scale to other types of models. In this
article we proposed the MinPEN framework to simultaneously estimate regression
coefficients associated with the multivariate regression model and the
relationships between outcome variables using mild assumptions. The MinPen
framework utilizes a novel penalty based on the minimum function to exploit
detected relationships between responses. An iterative algorithm that
generalizes current state of the art methods is proposed as a solution to the
non-convex optimization that is required to obtain estimates. Theoretical
results such as high dimensional convergence rates, model selection
consistency, and a framework for post selection inference are provided. We
extend the proposed MinPen framework to other exponential family loss
functions, with a specific focus on multiple binomial responses. Tuning
parameter selection is also addressed. Finally, simulations and two data
examples are presented to show the finite sample properties of this framework.
Related papers
- Semiparametric conformal prediction [79.6147286161434]
Risk-sensitive applications require well-calibrated prediction sets over multiple, potentially correlated target variables.
We treat the scores as random vectors and aim to construct the prediction set accounting for their joint correlation structure.
We report desired coverage and competitive efficiency on a range of real-world regression problems.
arXiv Detail & Related papers (2024-11-04T14:29:02Z) - Multivariate root-n-consistent smoothing parameter free matching estimators and estimators of inverse density weighted expectations [51.000851088730684]
We develop novel modifications of nearest-neighbor and matching estimators which converge at the parametric $sqrt n $-rate.
We stress that our estimators do not involve nonparametric function estimators and in particular do not rely on sample-size dependent parameters smoothing.
arXiv Detail & Related papers (2024-07-11T13:28:34Z) - Proximal Interacting Particle Langevin Algorithms [0.0]
We introduce Proximal Interacting Particle Langevin Algorithms (PIPLA) for inference and learning in latent variable models.
We propose several variants within the novel proximal IPLA family, tailored to the problem of estimating parameters in a non-differentiable statistical model.
Our theory and experiments together show that PIPLA family can be the de facto choice for parameter estimation problems in latent variable models for non-differentiable models.
arXiv Detail & Related papers (2024-06-20T13:16:41Z) - Multi-Response Heteroscedastic Gaussian Process Models and Their
Inference [1.52292571922932]
We propose a novel framework for the modeling of heteroscedastic covariance functions.
We employ variational inference to approximate the posterior and facilitate posterior predictive modeling.
We show that our proposed framework offers a robust and versatile tool for a wide array of applications.
arXiv Detail & Related papers (2023-08-29T15:06:47Z) - An ADMM approach for multi-response regression with overlapping groups
and interaction effects [0.0]
We propose MADMMplasso, a novel regularized regression method.
For parameter estimation, we develop an ADMM algorithm that allows us to implement the overlapping groups in a simple way.
The results from the simulations and analysis of a pharmacogenomic screen data set show that the proposed method has an advantage in handling correlated responses and interaction effects.
arXiv Detail & Related papers (2023-03-20T14:43:22Z) - Sparse high-dimensional linear regression with a partitioned empirical
Bayes ECM algorithm [62.997667081978825]
We propose a computationally efficient and powerful Bayesian approach for sparse high-dimensional linear regression.
Minimal prior assumptions on the parameters are used through the use of plug-in empirical Bayes estimates.
The proposed approach is implemented in the R package probe.
arXiv Detail & Related papers (2022-09-16T19:15:50Z) - Adaptive LASSO estimation for functional hidden dynamic geostatistical
model [69.10717733870575]
We propose a novel model selection algorithm based on a penalized maximum likelihood estimator (PMLE) for functional hiddenstatistical models (f-HD)
The algorithm is based on iterative optimisation and uses an adaptive least absolute shrinkage and selector operator (GMSOLAS) penalty function, wherein the weights are obtained by the unpenalised f-HD maximum-likelihood estimators.
arXiv Detail & Related papers (2022-08-10T19:17:45Z) - MINIMALIST: Mutual INformatIon Maximization for Amortized Likelihood
Inference from Sampled Trajectories [61.3299263929289]
Simulation-based inference enables learning the parameters of a model even when its likelihood cannot be computed in practice.
One class of methods uses data simulated with different parameters to infer an amortized estimator for the likelihood-to-evidence ratio.
We show that this approach can be formulated in terms of mutual information between model parameters and simulated data.
arXiv Detail & Related papers (2021-06-03T12:59:16Z) - Variable selection for Gaussian process regression through a sparse
projection [0.802904964931021]
This paper presents a new variable selection approach integrated with Gaussian process (GP) regression.
The choice of tuning parameters and the accuracy of the estimation are evaluated with the simulation some chosen benchmark approaches.
arXiv Detail & Related papers (2020-08-25T01:06:10Z) - Instability, Computational Efficiency and Statistical Accuracy [101.32305022521024]
We develop a framework that yields statistical accuracy based on interplay between the deterministic convergence rate of the algorithm at the population level, and its degree of (instability) when applied to an empirical object based on $n$ samples.
We provide applications of our general results to several concrete classes of models, including Gaussian mixture estimation, non-linear regression models, and informative non-response models.
arXiv Detail & Related papers (2020-05-22T22:30:52Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.