Sparse-group boosting -- Unbiased group and variable selection

• URL: http://arxiv.org/abs/2206.06344v2
• Date: Sun, 7 Apr 2024 12:32:23 GMT
• Title: Sparse-group boosting -- Unbiased group and variable selection
• Authors: Fabian Obster, Christian Heumann,
• Abstract summary: We show that within-group and between-group sparsity can be controlled by a mixing parameter. With simulations, gene data as well as agricultural data we show the effectiveness and predictive competitiveness of this estimator.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the presence of grouped covariates, we propose a framework for boosting that allows to enforce sparsity within and between groups. By using component-wise and group-wise gradient boosting at the same time with adjusted degrees of freedom, a model with similar properties as the sparse group lasso can be fitted through boosting. We show that within-group and between-group sparsity can be controlled by a mixing parameter and discuss similarities and differences to the mixing parameter in the sparse group lasso. With simulations, gene data as well as agricultural data we show the effectiveness and predictive competitiveness of this estimator. The data and simulations suggest, that in the presence of grouped variables the use of sparse group boosting is associated with less biased variable selection and higher predictability compared to component-wise boosting. Additionally, we propose a way of reducing bias in component-wise boosting through the degrees of freedom.

Related papers

• Introducing sgboost: A Practical Guide and Implementation of sparse-group boosting in R [0.0]
  This paper introduces the sgboost package in R, which implements sparse-group boosting for modeling high-dimensional data. The package uses regularization techniques based on the degrees of freedom of individual and group base-learners.
  arXiv  Detail & Related papers  (2024-05-31T17:29:51Z)
• GroupedMixer: An Entropy Model with Group-wise Token-Mixers for Learned Image Compression [64.47244912937204]
  We propose a novel transformer-based entropy model called GroupedMixer. GroupedMixer enjoys both faster coding speed and better compression performance than previous transformer-based methods. Experimental results demonstrate that the proposed GroupedMixer yields the state-of-the-art rate-distortion performance with fast compression speed.
  arXiv  Detail & Related papers  (2024-05-02T10:48:22Z)
• Affinity Clustering Framework for Data Debiasing Using Pairwise Distribution Discrepancy [10.184056098238765]
  Group imbalance, resulting from inadequate or unrepresentative data collection methods, is a primary cause of representation bias in datasets. This paper presents MASC, a data augmentation approach that leverages affinity clustering to balance the representation of non-protected and protected groups of a target dataset.
  arXiv  Detail & Related papers  (2023-06-02T17:18:20Z)
• Statistical Guarantees of Group-Invariant GANs [13.084804346845816]
  Group-invariant generative adversarial networks (GANs) are a type of GANs in which the generators and discriminators are hardwired with group symmetries. This work presents the first statistical performance guarantees for group-invariant generative models, specifically for GANs.
  arXiv  Detail & Related papers  (2023-05-22T22:13:12Z)
• HiPerformer: Hierarchically Permutation-Equivariant Transformer for Time Series Forecasting [56.95572957863576]
  We propose a hierarchically permutation-equivariant model that considers both the relationship among components in the same group and the relationship among groups. The experiments conducted on real-world data demonstrate that the proposed method outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-14T05:11:52Z)
• Fairness via Adversarial Attribute Neighbourhood Robust Learning [49.93775302674591]
  We propose a principled underlineRobust underlineAdversarial underlineAttribute underlineNeighbourhood (RAAN) loss to debias the classification head.
  arXiv  Detail & Related papers  (2022-10-12T23:39:28Z)
• Towards Group Robustness in the presence of Partial Group Labels [61.33713547766866]
  spurious correlations between input samples and the target labels wrongly direct the neural network predictions. We propose an algorithm that optimize for the worst-off group assignments from a constraint set. We show improvements in the minority group's performance while preserving overall aggregate accuracy across groups.
  arXiv  Detail & Related papers  (2022-01-10T22:04:48Z)
• Feature Grouping and Sparse Principal Component Analysis [23.657672812296518]
  Grouping and Sparse Principal Analysis (SPCA) is widely used in data processing dimension reduction. FGSPCA allows loadings to belong to disjoint homogeneous groups, with sparsity as a special case.
  arXiv  Detail & Related papers  (2021-06-25T15:08:39Z)
• Deconfounding Scores: Feature Representations for Causal Effect Estimation with Weak Overlap [140.98628848491146]
  We introduce deconfounding scores, which induce better overlap without biasing the target of estimation. We show that deconfounding scores satisfy a zero-covariance condition that is identifiable in observed data. In particular, we show that this technique could be an attractive alternative to standard regularizations.
  arXiv  Detail & Related papers  (2021-04-12T18:50:11Z)
• Contrastive Clustering [57.71729650297379]
  We propose Contrastive Clustering (CC) which explicitly performs the instance- and cluster-level contrastive learning. In particular, CC achieves an NMI of 0.705 (0.431) on the CIFAR-10 (CIFAR-100) dataset, which is an up to 19% (39%) performance improvement compared with the best baseline.
  arXiv  Detail & Related papers  (2020-09-21T08:54:40Z)
• Robust Grouped Variable Selection Using Distributionally Robust Optimization [11.383869751239166]
  We propose a Distributionally Robust Optimization (DRO) formulation with a Wasserstein-based uncertainty set for selecting grouped variables under perturbations. We prove probabilistic bounds on the out-of-sample loss and the estimation bias, and establish the grouping effect of our estimator. We show that our formulation produces an interpretable and parsimonious model that encourages sparsity at a group level.
  arXiv  Detail & Related papers  (2020-06-10T22:32:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.