Fréchet regression for multi-label feature selection with implicit regularization

• URL: http://arxiv.org/abs/2412.18247v1
• Date: Tue, 24 Dec 2024 08:02:28 GMT
• Title: Fréchet regression for multi-label feature selection with implicit regularization
• Authors: Dou El Kefel Mansouri, Seif-Eddine Benkabou, Khalid Benabdeslem,
• Abstract summary: We propose a novel variable selection method that employs implicit regularization instead of traditional explicit regularization approaches. Our method effectively captures nonlinear interactions between predic tors and responses while promoting model sparsity.
• Score: 1.5771347525430772
• License:
• Abstract: Fr\'echet regression extends linear regression to model complex responses in metric spaces, making it particularly relevant for multi-label regression, where each instance can have multiple associated labels. However, variable selection within this framework remains underexplored. In this paper, we pro pose a novel variable selection method that employs implicit regularization instead of traditional explicit regularization approaches, which can introduce bias. Our method effectively captures nonlinear interactions between predic tors and responses while promoting model sparsity. We provide theoretical results demonstrating selection consistency and illustrate the performance of our approach through numerical examples

Related papers

• GeneralizeFormer: Layer-Adaptive Model Generation across Test-Time Distribution Shifts [58.95913531746308]
  We consider the problem of test-time domain generalization, where a model is trained on several source domains and adjusted on target domains never seen during training. We propose to generate multiple layer parameters on the fly during inference by a lightweight meta-learned transformer, which we call textitGeneralizeFormer.
  arXiv  Detail & Related papers  (2025-02-15T10:10:49Z)
• Variational empirical Bayes variable selection in high-dimensional logistic regression [2.4032899110671955]
  We develop a novel and computationally efficient variational approximation thereof. One such novelty is that we develop this approximation directly for the marginal distribution on the model space, rather than on the regression coefficients themselves. We demonstrate the method's strong performance in simulations, and prove that our variational approximation inherits the strong selection consistency property satisfied by the posterior distribution that it is approximating.
  arXiv  Detail & Related papers  (2025-02-14T19:57:13Z)
• Adaptive Collaborative Correlation Learning-based Semi-Supervised Multi-Label Feature Selection [25.195711274756334]
  We propose an Adaptive Collaborative Correlation lEarning-based Semi-Supervised Multi-label Feature Selection (Access-MFS) method to address these issues. Specifically, a generalized regression model equipped with an extended uncorrelated constraint is introduced to select discriminative yet irrelevant features. The correlation instance and label correlation are integrated into the proposed regression model to adaptively learn both the sample similarity graph and the label similarity graph.
  arXiv  Detail & Related papers  (2024-06-18T01:47:38Z)
• Selective Nonparametric Regression via Testing [54.20569354303575]
  We develop an abstention procedure via testing the hypothesis on the value of the conditional variance at a given point. Unlike existing methods, the proposed one allows to account not only for the value of the variance itself but also for the uncertainty of the corresponding variance predictor.
  arXiv  Detail & Related papers  (2023-09-28T13:04:11Z)
• Correcting for Selection Bias and Missing Response in Regression using Privileged Information [1.5049442691806052]
  We propose a novel imputation-based regression method, named repeated regression, that is suitable for Privilegedly Missing at Random (PMAR) We empirically assess the performance of the proposed methods with extensive simulated experiments and on a synthetically augmented real-world dataset.
  arXiv  Detail & Related papers  (2023-03-29T15:42:17Z)
• Flexible variable selection in the presence of missing data [0.0]
  We propose a non-parametric variable selection algorithm combined with multiple imputation to develop flexible panels in the presence of missing-at-random data. We show that our proposal has good operating characteristics and results in panels with higher classification and variable selection performance.
  arXiv  Detail & Related papers  (2022-02-25T21:41:03Z)
• Machine Learning for Multi-Output Regression: When should a holistic multivariate approach be preferred over separate univariate ones? [62.997667081978825]
  Tree-based ensembles such as the Random Forest are modern classics among statistical learning methods. We compare these methods in extensive simulations to help in answering the primary question when to use multivariate ensemble techniques.
  arXiv  Detail & Related papers  (2022-01-14T08:44:25Z)
• Time varying regression with hidden linear dynamics [74.9914602730208]
  We revisit a model for time-varying linear regression that assumes the unknown parameters evolve according to a linear dynamical system. Counterintuitively, we show that when the underlying dynamics are stable the parameters of this model can be estimated from data by combining just two ordinary least squares estimates.
  arXiv  Detail & Related papers  (2021-12-29T23:37:06Z)
• Causally-motivated Shortcut Removal Using Auxiliary Labels [63.686580185674195]
  Key challenge to learning such risk-invariant predictors is shortcut learning. We propose a flexible, causally-motivated approach to address this challenge. We show both theoretically and empirically that this causally-motivated regularization scheme yields robust predictors.
  arXiv  Detail & Related papers  (2021-05-13T16:58:45Z)
• Nonlinear Independent Component Analysis for Continuous-Time Signals [85.59763606620938]
  We study the classical problem of recovering a multidimensional source process from observations of mixtures of this process. We show that this recovery is possible for many popular models of processes (up to order and monotone scaling of their coordinates) if the mixture is given by a sufficiently differentiable, invertible function.
  arXiv  Detail & Related papers  (2021-02-04T20:28:44Z)
• Reduced-Rank Tensor-on-Tensor Regression and Tensor-variate Analysis of Variance [11.193504036335503]
  We extend the classical multivariate regression model to exploit such structure. We obtain maximum likelihood estimators via block-relaxation algorithms. A separate application performs three-way TANOVA on the Labeled Faces in the Wild image database.
  arXiv  Detail & Related papers  (2020-12-18T14:04:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.