A Boosting Approach to Constructing an Ensemble Stack
- URL: http://arxiv.org/abs/2211.15621v1
- Date: Mon, 28 Nov 2022 18:21:36 GMT
- Title: A Boosting Approach to Constructing an Ensemble Stack
- Authors: Zhilei Zhou and Ziyu Qiu and Brad Niblett and Andrew Johnston and
Jeffrey Schwartzentruber and Nur Zincir-Heywood and Malcolm Heywood
- Abstract summary: An approach to evolutionary ensemble learning for classification is proposed in which boosting is used to construct a stack of programs.
Training against a residual dataset actively reduces the cost of training.
Benchmarking studies are conducted to illustrate competitiveness with the prediction accuracy of current state-of-the-art evolutionary ensemble learning algorithms.
- Score: 1.0775419935941009
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: An approach to evolutionary ensemble learning for classification is proposed
in which boosting is used to construct a stack of programs. Each application of
boosting identifies a single champion and a residual dataset, i.e. the training
records that thus far were not correctly classified. The next program is only
trained against the residual, with the process iterating until some maximum
ensemble size or no further residual remains. Training against a residual
dataset actively reduces the cost of training. Deploying the ensemble as a
stack also means that only one classifier might be necessary to make a
prediction, so improving interpretability. Benchmarking studies are conducted
to illustrate competitiveness with the prediction accuracy of current
state-of-the-art evolutionary ensemble learning algorithms, while providing
solutions that are orders of magnitude simpler. Further benchmarking with a
high cardinality dataset indicates that the proposed method is also more
accurate and efficient than XGBoost.
Related papers
- Classification Tree-based Active Learning: A Wrapper Approach [4.706932040794696]
This paper proposes a wrapper active learning method for classification, organizing the sampling process into a tree structure.
A classification tree constructed on an initial set of labeled samples is considered to decompose the space into low-entropy regions.
This adaptation proves to be a significant enhancement over existing active learning methods.
arXiv Detail & Related papers (2024-04-15T17:27:00Z) - Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple
Logits Retargeting Approach [102.0769560460338]
We develop a simple logits approach (LORT) without the requirement of prior knowledge of the number of samples per class.
Our method achieves state-of-the-art performance on various imbalanced datasets, including CIFAR100-LT, ImageNet-LT, and iNaturalist 2018.
arXiv Detail & Related papers (2024-03-01T03:27:08Z) - A Hard-to-Beat Baseline for Training-free CLIP-based Adaptation [121.0693322732454]
Contrastive Language-Image Pretraining (CLIP) has gained popularity for its remarkable zero-shot capacity.
Recent research has focused on developing efficient fine-tuning methods to enhance CLIP's performance in downstream tasks.
We revisit a classical algorithm, Gaussian Discriminant Analysis (GDA), and apply it to the downstream classification of CLIP.
arXiv Detail & Related papers (2024-02-06T15:45:27Z) - One-bit Supervision for Image Classification: Problem, Solution, and
Beyond [114.95815360508395]
This paper presents one-bit supervision, a novel setting of learning with fewer labels, for image classification.
We propose a multi-stage training paradigm and incorporate negative label suppression into an off-the-shelf semi-supervised learning algorithm.
In multiple benchmarks, the learning efficiency of the proposed approach surpasses that using full-bit, semi-supervised supervision.
arXiv Detail & Related papers (2023-11-26T07:39:00Z) - XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners [71.8257151788923]
We propose a novel Explainable Active Learning framework (XAL) for low-resource text classification.
XAL encourages classifiers to justify their inferences and delve into unlabeled data for which they cannot provide reasonable explanations.
Experiments on six datasets show that XAL achieves consistent improvement over 9 strong baselines.
arXiv Detail & Related papers (2023-10-09T08:07:04Z) - Revisiting Long-tailed Image Classification: Survey and Benchmarks with
New Evaluation Metrics [88.39382177059747]
A corpus of metrics is designed for measuring the accuracy, robustness, and bounds of algorithms for learning with long-tailed distribution.
Based on our benchmarks, we re-evaluate the performance of existing methods on CIFAR10 and CIFAR100 datasets.
arXiv Detail & Related papers (2023-02-03T02:40:54Z) - SetConv: A New Approach for Learning from Imbalanced Data [29.366843553056594]
We propose a set convolution operation and an episodic training strategy to extract a single representative for each class.
We prove that our proposed algorithm is permutation-invariant despite the order of inputs.
arXiv Detail & Related papers (2021-04-03T22:33:30Z) - Uncovering Coresets for Classification With Multi-Objective Evolutionary
Algorithms [0.8057006406834467]
A coreset is a subset of the training set, using which a machine learning algorithm obtains performances similar to what it would deliver if trained over the whole original data.
A novel approach is presented: candidate corsets are iteratively optimized, adding and removing samples.
A multi-objective evolutionary algorithm is used to minimize simultaneously the number of points in the set and the classification error.
arXiv Detail & Related papers (2020-02-20T09:59:56Z) - Progressive Identification of True Labels for Partial-Label Learning [112.94467491335611]
Partial-label learning (PLL) is a typical weakly supervised learning problem, where each training instance is equipped with a set of candidate labels among which only one is the true label.
Most existing methods elaborately designed as constrained optimizations that must be solved in specific manners, making their computational complexity a bottleneck for scaling up to big data.
This paper proposes a novel framework of classifier with flexibility on the model and optimization algorithm.
arXiv Detail & Related papers (2020-02-19T08:35:15Z) - ScaIL: Classifier Weights Scaling for Class Incremental Learning [12.657788362927834]
In a deep learning approach, the constant computational budget requires the use of a fixed architecture for all incremental states.
The bounded memory generates data imbalance in favor of new classes and a prediction bias toward them appears.
We propose simple but efficient scaling of past class classifier weights to make them more comparable to those of new classes.
arXiv Detail & Related papers (2020-01-16T12:10:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.