Exclusive Group Lasso for Structured Variable Selection

• URL: http://arxiv.org/abs/2108.10284v2
• Date: Thu, 2 Nov 2023 07:05:56 GMT
• Title: Exclusive Group Lasso for Structured Variable Selection
• Authors: David Gregoratti and Xavier Mestre and Carlos Buelga
• Abstract summary: A structured variable selection problem is considered. A composite norm can be properly designed to promote such exclusive group sparsity patterns. An active set algorithm is proposed that builds the solution by including structure atoms into the estimated support.
• Score: 10.86544864007391
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A structured variable selection problem is considered in which the covariates, divided into predefined groups, activate according to sparse patterns with few nonzero entries per group. Capitalizing on the concept of atomic norm, a composite norm can be properly designed to promote such exclusive group sparsity patterns. The resulting norm lends itself to efficient and flexible regularized optimization algorithms for support recovery, like the proximal algorithm. Moreover, an active set algorithm is proposed that builds the solution by successively including structure atoms into the estimated support. It is also shown that such an algorithm can be tailored to match more rigid structures than plain exclusive group sparsity. Asymptotic consistency analysis (with both the number of parameters as well as the number of groups growing with the observation size) establishes the effectiveness of the proposed solution in terms of signed support recovery under conventional assumptions. Finally, a set of numerical simulations further corroborates the results.

Related papers

• Sparsity via Sparse Group $k$-max Regularization [22.05774771336432]
  In this paper, we propose a novel and concise regularization, namely the sparse group $k$-max regularization. We verify the effectiveness and flexibility of the proposed method through numerical experiments on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2024-02-13T14:41:28Z)
• Likelihood Ratio Confidence Sets for Sequential Decision Making [51.66638486226482]
  We revisit the likelihood-based inference principle and propose to use likelihood ratios to construct valid confidence sequences. Our method is especially suitable for problems with well-specified likelihoods. We show how to provably choose the best sequence of estimators and shed light on connections to online convex optimization.
  arXiv  Detail & Related papers  (2023-11-08T00:10:21Z)
• HiPerformer: Hierarchically Permutation-Equivariant Transformer for Time Series Forecasting [56.95572957863576]
  We propose a hierarchically permutation-equivariant model that considers both the relationship among components in the same group and the relationship among groups. The experiments conducted on real-world data demonstrate that the proposed method outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-14T05:11:52Z)
• Group conditional validity via multi-group learning [5.797821810358083]
  We consider the problem of distribution-free conformal prediction and the criterion of group conditional validity. Existing methods achieve such guarantees under either restrictive grouping structure or distributional assumptions. We propose a simple reduction to the problem of achieving validity guarantees for individual populations by leveraging algorithms for a problem called multi-group learning.
  arXiv  Detail & Related papers  (2023-03-07T15:51:03Z)
• Linearization Algorithms for Fully Composite Optimization [61.20539085730636]
  This paper studies first-order algorithms for solving fully composite optimization problems convex compact sets. We leverage the structure of the objective by handling differentiable and non-differentiable separately, linearizing only the smooth parts.
  arXiv  Detail & Related papers  (2023-02-24T18:41:48Z)
• Revisiting GANs by Best-Response Constraint: Perspective, Methodology, and Application [49.66088514485446]
  Best-Response Constraint (BRC) is a general learning framework to explicitly formulate the potential dependency of the generator on the discriminator. We show that even with different motivations and formulations, a variety of existing GANs ALL can be uniformly improved by our flexible BRC methodology.
  arXiv  Detail & Related papers  (2022-05-20T12:42:41Z)
• Feature Grouping and Sparse Principal Component Analysis [23.657672812296518]
  Grouping and Sparse Principal Analysis (SPCA) is widely used in data processing dimension reduction. FGSPCA allows loadings to belong to disjoint homogeneous groups, with sparsity as a special case.
  arXiv  Detail & Related papers  (2021-06-25T15:08:39Z)
• Certifiably Polynomial Algorithm for Best Group Subset Selection [0.9667631210393929]
  Best group subset selection aims to choose a small part of non-overlapping groups to achieve the best interpretability on the response variable. We propose a group-splicing algorithm that iteratively detects effective groups and excludes the helpless ones. We demonstrate the efficiency and accuracy of our proposal by comparing state-of-the-art algorithms on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2021-04-23T03:05:11Z)
• GroupifyVAE: from Group-based Definition to VAE-based Unsupervised Representation Disentanglement [91.9003001845855]
  VAE-based unsupervised disentanglement can not be achieved without introducing other inductive bias. We address VAE-based unsupervised disentanglement by leveraging the constraints derived from the Group Theory based definition as the non-probabilistic inductive bias. We train 1800 models covering the most prominent VAE-based models on five datasets to verify the effectiveness of our method.
  arXiv  Detail & Related papers  (2021-02-20T09:49:51Z)
• Efficient Methods for Structured Nonconvex-Nonconcave Min-Max Optimization [98.0595480384208]
  We propose a generalization extraient spaces which converges to a stationary point. The algorithm applies not only to general $p$-normed spaces, but also to general $p$-dimensional vector spaces.
  arXiv  Detail & Related papers  (2020-10-31T21:35:42Z)
• Robust Grouped Variable Selection Using Distributionally Robust Optimization [11.383869751239166]
  We propose a Distributionally Robust Optimization (DRO) formulation with a Wasserstein-based uncertainty set for selecting grouped variables under perturbations. We prove probabilistic bounds on the out-of-sample loss and the estimation bias, and establish the grouping effect of our estimator. We show that our formulation produces an interpretable and parsimonious model that encourages sparsity at a group level.
  arXiv  Detail & Related papers  (2020-06-10T22:32:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.