DivBO: Diversity-aware CASH for Ensemble Learning
- URL: http://arxiv.org/abs/2302.03255v1
- Date: Tue, 7 Feb 2023 04:53:21 GMT
- Title: DivBO: Diversity-aware CASH for Ensemble Learning
- Authors: Yu Shen, Yupeng Lu, Yang Li, Yaofeng Tu, Wentao Zhang, Bin Cui
- Abstract summary: We propose DivBO, a diversity-aware framework to inject explicit search of diversity into the CASH problems.
In the framework, we propose to use a diversity surrogate to predict the pair-wise diversity of two unseen configurations.
We show that DivBO achieves the best average ranks (1.82 and 1.73) on both validation and test errors among 10 compared methods.
- Score: 26.18421492435029
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The Combined Algorithm Selection and Hyperparameters optimization (CASH)
problem is one of the fundamental problems in Automated Machine Learning
(AutoML). Motivated by the success of ensemble learning, recent AutoML systems
build post-hoc ensembles to output the final predictions instead of using the
best single learner. However, while most CASH methods focus on searching for a
single learner with the best performance, they neglect the diversity among base
learners (i.e., they may suggest similar configurations to previously evaluated
ones), which is also a crucial consideration when building an ensemble. To
tackle this issue and further enhance the ensemble performance, we propose
DivBO, a diversity-aware framework to inject explicit search of diversity into
the CASH problems. In the framework, we propose to use a diversity surrogate to
predict the pair-wise diversity of two unseen configurations. Furthermore, we
introduce a temporary pool and a weighted acquisition function to guide the
search of both performance and diversity based on Bayesian optimization.
Empirical results on 15 public datasets show that DivBO achieves the best
average ranks (1.82 and 1.73) on both validation and test errors among 10
compared methods, including post-hoc designs in recent AutoML systems and
state-of-the-art baselines for ensemble learning on CASH problems.
Related papers
- POGEMA: A Benchmark Platform for Cooperative Multi-Agent Navigation [76.67608003501479]
We introduce and specify an evaluation protocol defining a range of domain-related metrics computed on the basics of the primary evaluation indicators.
The results of such a comparison, which involves a variety of state-of-the-art MARL, search-based, and hybrid methods, are presented.
arXiv Detail & Related papers (2024-07-20T16:37:21Z) - UCB-driven Utility Function Search for Multi-objective Reinforcement Learning [75.11267478778295]
In Multi-objective Reinforcement Learning (MORL) agents are tasked with optimising decision-making behaviours.
We focus on the case of linear utility functions parameterised by weight vectors w.
We introduce a method based on Upper Confidence Bound to efficiently search for the most promising weight vectors during different stages of the learning process.
arXiv Detail & Related papers (2024-05-01T09:34:42Z) - Zero-shot Task Preference Addressing Enabled by Imprecise Bayesian
Continual Learning [19.11678487931003]
We propose Imprecise Bayesian Continual Learning (IBCL) to address preferences on task-performance trade-offs.
IBCL does not require any additional training overhead to construct preference-addressing models from its knowledge base.
We show that models obtained by IBCL have guarantees in identifying the preferred parameters.
arXiv Detail & Related papers (2023-05-24T06:39:00Z) - Deep Negative Correlation Classification [82.45045814842595]
Existing deep ensemble methods naively train many different models and then aggregate their predictions.
We propose deep negative correlation classification (DNCC)
DNCC yields a deep classification ensemble where the individual estimator is both accurate and negatively correlated.
arXiv Detail & Related papers (2022-12-14T07:35:20Z) - Learning with MISELBO: The Mixture Cookbook [62.75516608080322]
We present the first ever mixture of variational approximations for a normalizing flow-based hierarchical variational autoencoder (VAE) with VampPrior and a PixelCNN decoder network.
We explain this cooperative behavior by drawing a novel connection between VI and adaptive importance sampling.
We obtain state-of-the-art results among VAE architectures in terms of negative log-likelihood on the MNIST and FashionMNIST datasets.
arXiv Detail & Related papers (2022-09-30T15:01:35Z) - Developing parsimonious ensembles using predictor diversity within a
reinforcement learning framework [4.204145943086225]
We present several algorithms that incorporate ensemble diversity into a reinforcement learning (RL)-based ensemble selection framework.
These algorithms can eventually aid the interpretation or reverse engineering of predictive models assimilated into effective ensembles.
arXiv Detail & Related papers (2021-02-15T05:00:19Z) - Towards Model-Agnostic Post-Hoc Adjustment for Balancing Ranking
Fairness and Algorithm Utility [54.179859639868646]
Bipartite ranking aims to learn a scoring function that ranks positive individuals higher than negative ones from labeled data.
There have been rising concerns on whether the learned scoring function can cause systematic disparity across different protected groups.
We propose a model post-processing framework for balancing them in the bipartite ranking scenario.
arXiv Detail & Related papers (2020-06-15T10:08:39Z) - Bayesian Meta-Prior Learning Using Empirical Bayes [3.666114237131823]
We propose a hierarchical Empirical Bayes approach that addresses the absence of informative priors, and the inability to control parameter learning rates.
Our method learns empirical meta-priors from the data itself and uses them to decouple the learning rates of first-order and second-order features.
Our findings are promising, as optimizing over sparse data is often a challenge.
arXiv Detail & Related papers (2020-02-04T05:08:17Z) - Stepwise Model Selection for Sequence Prediction via Deep Kernel
Learning [100.83444258562263]
We propose a novel Bayesian optimization (BO) algorithm to tackle the challenge of model selection in this setting.
In order to solve the resulting multiple black-box function optimization problem jointly and efficiently, we exploit potential correlations among black-box functions.
We are the first to formulate the problem of stepwise model selection (SMS) for sequence prediction, and to design and demonstrate an efficient joint-learning algorithm for this purpose.
arXiv Detail & Related papers (2020-01-12T09:42:19Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.