Vector-Valued Least-Squares Regression under Output Regularity Assumptions

• URL: http://arxiv.org/abs/2211.08958v1
• Date: Wed, 16 Nov 2022 15:07:00 GMT
• Title: Vector-Valued Least-Squares Regression under Output Regularity Assumptions
• Authors: Luc Brogat-Motte, Alessandro Rudi, C\'eline Brouard, Juho Rousu, Florence d'Alch\'e-Buc
• Abstract summary: We propose and analyse a reduced-rank method for solving least-squares regression problems with infinite dimensional output. We derive learning bounds for our method, and study under which setting statistical performance is improved in comparison to full-rank method.
• Score: 73.99064151691597
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose and analyse a reduced-rank method for solving least-squares regression problems with infinite dimensional output. We derive learning bounds for our method, and study under which setting statistical performance is improved in comparison to full-rank method. Our analysis extends the interest of reduced-rank regression beyond the standard low-rank setting to more general output regularity assumptions. We illustrate our theoretical insights on synthetic least-squares problems. Then, we propose a surrogate structured prediction method derived from this reduced-rank method. We assess its benefits on three different problems: image reconstruction, multi-label classification, and metabolite identification.

Related papers

• Distributed High-Dimensional Quantile Regression: Estimation Efficiency and Support Recovery [0.0]
  We focus on distributed estimation and support recovery for high-dimensional linear quantile regression. We transform the original quantile regression into the least-squares optimization. An efficient algorithm is developed, which enjoys high computation and communication efficiency.
  arXiv  Detail & Related papers  (2024-05-13T08:32:22Z)
• Analysis of Bootstrap and Subsampling in High-dimensional Regularized Regression [29.57766164934947]
  We investigate popular resampling methods for estimating the uncertainty of statistical models. We provide a tight description of the biases and variances estimated by these methods in the context of generalized linear models.
  arXiv  Detail & Related papers  (2024-02-21T08:50:33Z)
• Transfer Learning for Nonparametric Regression: Non-asymptotic Minimax Analysis and Adaptive Procedure [5.303044915173525]
  We develop a novel estimator called the confidence thresholding estimator, which is shown to achieve the minimax optimal risk up to a logarithmic factor. We then propose a data-driven algorithm that adaptively achieves the minimax risk up to a logarithmic factor across a wide range of parameter spaces.
  arXiv  Detail & Related papers  (2024-01-22T16:24:04Z)
• Selective Nonparametric Regression via Testing [54.20569354303575]
  We develop an abstention procedure via testing the hypothesis on the value of the conditional variance at a given point. Unlike existing methods, the proposed one allows to account not only for the value of the variance itself but also for the uncertainty of the corresponding variance predictor.
  arXiv  Detail & Related papers  (2023-09-28T13:04:11Z)
• Errors-in-variables Fr\'echet Regression with Low-rank Covariate Approximation [2.1756081703276]
  Fr'echet regression has emerged as a promising approach for regression analysis involving non-Euclidean response variables. Our proposed framework combines the concepts of global Fr'echet regression and principal component regression, aiming to improve the efficiency and accuracy of the regression estimator.
  arXiv  Detail & Related papers  (2023-05-16T08:37:54Z)
• Mitigating multiple descents: A model-agnostic framework for risk monotonization [84.6382406922369]
  We develop a general framework for risk monotonization based on cross-validation. We propose two data-driven methodologies, namely zero- and one-step, that are akin to bagging and boosting.
  arXiv  Detail & Related papers  (2022-05-25T17:41:40Z)
• Robust Regularized Low-Rank Matrix Models for Regression and Classification [14.698622796774634]
  We propose a framework for matrix variate regression models based on a rank constraint, vector regularization (e.g., sparsity), and a general loss function. We show that the algorithm is guaranteed to converge, all accumulation points of the algorithm have estimation errors in the order of $O(sqrtn)$ally and substantially attaining the minimax rate.
  arXiv  Detail & Related papers  (2022-05-14T18:03:48Z)
• Fast OSCAR and OWL Regression via Safe Screening Rules [97.28167655721766]
  Ordered $L_1$ (OWL) regularized regression is a new regression analysis for high-dimensional sparse learning. Proximal gradient methods are used as standard approaches to solve OWL regression. We propose the first safe screening rule for OWL regression by exploring the order of the primal solution with the unknown order structure.
  arXiv  Detail & Related papers  (2020-06-29T23:35:53Z)
• Compressing Large Sample Data for Discriminant Analysis [78.12073412066698]
  We consider the computational issues due to large sample size within the discriminant analysis framework. We propose a new compression approach for reducing the number of training samples for linear and quadratic discriminant analysis.
  arXiv  Detail & Related papers  (2020-05-08T05:09:08Z)
• The Simulator: Understanding Adaptive Sampling in the Moderate-Confidence Regime [52.38455827779212]
  We propose a novel technique for analyzing adaptive sampling called the em Simulator. We prove the first instance-based lower bounds the top-k problem which incorporate the appropriate log-factors. Our new analysis inspires a simple and near-optimal for the best-arm and top-k identification, the first em practical of its kind for the latter problem.
  arXiv  Detail & Related papers  (2017-02-16T23:42:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.