Subgroup Robustness Grows On Trees: An Empirical Baseline Investigation
- URL: http://arxiv.org/abs/2211.12703v2
- Date: Mon, 17 Apr 2023 15:54:03 GMT
- Title: Subgroup Robustness Grows On Trees: An Empirical Baseline Investigation
- Authors: Josh Gardner, Zoran Popovi\'c, Ludwig Schmidt
- Abstract summary: We conduct an empirical comparison of several previously-proposed methods for fair and robust learning alongside state-of-the-art tree-based methods.
We show that tree-based methods have strong subgroup robustness, even when compared to robustness- and fairness-enhancing methods.
- Score: 13.458414200958797
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Researchers have proposed many methods for fair and robust machine learning,
but comprehensive empirical evaluation of their subgroup robustness is lacking.
In this work, we address this gap in the context of tabular data, where
sensitive subgroups are clearly-defined, real-world fairness problems abound,
and prior works often do not compare to state-of-the-art tree-based models as
baselines. We conduct an empirical comparison of several previously-proposed
methods for fair and robust learning alongside state-of-the-art tree-based
methods and other baselines. Via experiments with more than $340{,}000$ model
configurations on eight datasets, we show that tree-based methods have strong
subgroup robustness, even when compared to robustness- and fairness-enhancing
methods. Moreover, the best tree-based models tend to show good performance
over a range of metrics, while robust or group-fair models can show
brittleness, with significant performance differences across different metrics
for a fixed model. We also demonstrate that tree-based models show less
sensitivity to hyperparameter configurations, and are less costly to train. Our
work suggests that tree-based ensemble models make an effective baseline for
tabular data, and are a sensible default when subgroup robustness is desired.
For associated code and detailed results, see
https://github.com/jpgard/subgroup-robustness-grows-on-trees .
Related papers
- Modern Neighborhood Components Analysis: A Deep Tabular Baseline Two Decades Later [59.88557193062348]
We revisit the classic Neighborhood Component Analysis (NCA), designed to learn a linear projection that captures semantic similarities between instances.
We find that minor modifications, such as adjustments to the learning objectives and the integration of deep learning architectures, significantly enhance NCA's performance.
We also introduce a neighbor sampling strategy that improves both the efficiency and predictive accuracy of our proposed ModernNCA.
arXiv Detail & Related papers (2024-07-03T16:38:57Z) - Hierarchical clustering with dot products recovers hidden tree structure [53.68551192799585]
In this paper we offer a new perspective on the well established agglomerative clustering algorithm, focusing on recovery of hierarchical structure.
We recommend a simple variant of the standard algorithm, in which clusters are merged by maximum average dot product and not, for example, by minimum distance or within-cluster variance.
We demonstrate that the tree output by this algorithm provides a bona fide estimate of generative hierarchical structure in data, under a generic probabilistic graphical model.
arXiv Detail & Related papers (2023-05-24T11:05:12Z) - Why do tree-based models still outperform deep learning on tabular data? [0.0]
We show that tree-based models remain state-of-the-art on medium-sized data.
We conduct an empirical investigation into the differing inductive biases of tree-based models and Neural Networks (NNs)
arXiv Detail & Related papers (2022-07-18T08:36:08Z) - Explainable Models via Compression of Tree Ensembles [23.790618990173]
We consider the problem of compressing a large set of learned trees into a single explainable model.
CoTE -- Compression of Tree Ensembles -- produces a single small decision list as a compressed representation.
An experimental evaluation demonstrates the effectiveness of CoTE in several benchmark relational data sets.
arXiv Detail & Related papers (2022-06-16T04:03:55Z) - Scalable Whitebox Attacks on Tree-based Models [2.3186641356561646]
This paper proposes a novel whitebox adversarial robustness testing approach for tree ensemble models.
By leveraging sampling and the log-derivative trick, the proposed approach can scale up to testing tasks that were previously unmanageable.
arXiv Detail & Related papers (2022-03-31T21:36:20Z) - Clustering Effect of (Linearized) Adversarial Robust Models [60.25668525218051]
We propose a novel understanding of adversarial robustness and apply it on more tasks including domain adaption and robustness boosting.
Experimental evaluations demonstrate the rationality and superiority of our proposed clustering strategy.
arXiv Detail & Related papers (2021-11-25T05:51:03Z) - Sparse MoEs meet Efficient Ensembles [49.313497379189315]
We study the interplay of two popular classes of such models: ensembles of neural networks and sparse mixture of experts (sparse MoEs)
We present Efficient Ensemble of Experts (E$3$), a scalable and simple ensemble of sparse MoEs that takes the best of both classes of models, while using up to 45% fewer FLOPs than a deep ensemble.
arXiv Detail & Related papers (2021-10-07T11:58:35Z) - Dive into Decision Trees and Forests: A Theoretical Demonstration [0.0]
Decision trees use the strategy of "divide-and-conquer" to divide a complex problem on the dependency between input features and labels into smaller ones.
Recent advances have greatly improved their performance in computational advertising, recommender system, information retrieval, etc.
arXiv Detail & Related papers (2021-01-20T16:47:59Z) - MurTree: Optimal Classification Trees via Dynamic Programming and Search [61.817059565926336]
We present a novel algorithm for learning optimal classification trees based on dynamic programming and search.
Our approach uses only a fraction of the time required by the state-of-the-art and can handle datasets with tens of thousands of instances.
arXiv Detail & Related papers (2020-07-24T17:06:55Z) - Exploiting Syntactic Structure for Better Language Modeling: A Syntactic
Distance Approach [78.77265671634454]
We make use of a multi-task objective, i.e., the models simultaneously predict words as well as ground truth parse trees in a form called "syntactic distances"
Experimental results on the Penn Treebank and Chinese Treebank datasets show that when ground truth parse trees are provided as additional training signals, the model is able to achieve lower perplexity and induce trees with better quality.
arXiv Detail & Related papers (2020-05-12T15:35:00Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.