Divide, Specialize, and Route: A New Approach to Efficient Ensemble Learning

• URL: http://arxiv.org/abs/2506.20814v1
• Date: Wed, 25 Jun 2025 20:26:04 GMT
• Title: Divide, Specialize, and Route: A New Approach to Efficient Ensemble Learning
• Authors: Jakub Piwko, Jędrzej Ruciński, Dawid Płudowski, Antoni Zajko, Patryzja Żak, Mateusz Zacharecki, Anna Kozak, Katarzyna Woźnica,
• Abstract summary: We propose Hellsemble, a novel ensemble framework for binary classification.<n>Hellsemble incrementally partitions the dataset into circles of difficulty.<n>It achieves strong classification accuracy while maintaining computational efficiency and interpretability.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Ensemble learning has proven effective in boosting predictive performance, but traditional methods such as bagging, boosting, and dynamic ensemble selection (DES) suffer from high computational cost and limited adaptability to heterogeneous data distributions. To address these limitations, we propose Hellsemble, a novel and interpretable ensemble framework for binary classification that leverages dataset complexity during both training and inference. Hellsemble incrementally partitions the dataset into circles of difficulty by iteratively passing misclassified instances from simpler models to subsequent ones, forming a committee of specialised base learners. Each model is trained on increasingly challenging subsets, while a separate router model learns to assign new instances to the most suitable base model based on inferred difficulty. Hellsemble achieves strong classification accuracy while maintaining computational efficiency and interpretability. Experimental results on OpenML-CC18 and Tabzilla benchmarks demonstrate that Hellsemble often outperforms classical ensemble methods. Our findings suggest that embracing instance-level difficulty offers a promising direction for constructing efficient and robust ensemble systems.

Related papers

• Your Pretrained Model Tells the Difficulty Itself: A Self-Adaptive Curriculum Learning Paradigm for Natural Language Understanding [53.63482987410292]
  We present a self-adaptive curriculum learning paradigm that prioritizes fine-tuning examples based on difficulty scores predicted by pre-trained language models.<n>We evaluate our method on four natural language understanding (NLU) datasets covering both binary and multi-class classification tasks.
  arXiv  Detail & Related papers  (2025-07-13T19:36:17Z)
• Deep Learning Meets Oversampling: A Learning Framework to Handle Imbalanced Classification [0.0]
  We propose a novel learning framework that can generate synthetic data instances in a data-driven manner.<n>The proposed framework formulates the oversampling process as a composition of discrete decision criteria.<n>Experiments on the imbalanced classification task demonstrate the superiority of our framework over state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2025-02-08T13:35:00Z)
• Regularized Neural Ensemblers [55.15643209328513]
  In this study, we explore employing regularized neural networks as ensemble methods.<n>Motivated by the risk of learning low-diversity ensembles, we propose regularizing the ensembling model by randomly dropping base model predictions.<n>We demonstrate this approach provides lower bounds for the diversity within the ensemble, reducing overfitting and improving generalization capabilities.
  arXiv  Detail & Related papers  (2024-10-06T15:25:39Z)
• Proof of Swarm Based Ensemble Learning for Federated Learning Applications [3.2536767864585663]
  In federated learning it is not feasible to apply centralised ensemble learning directly due to privacy concerns. Most distributed consensus algorithms, such as Byzantine fault tolerance (BFT), do not normally perform well in such applications. We propose PoSw, a novel distributed consensus algorithm for ensemble learning in a federated setting.
  arXiv  Detail & Related papers  (2022-12-28T13:53:34Z)
• Deep Negative Correlation Classification [82.45045814842595]
  Existing deep ensemble methods naively train many different models and then aggregate their predictions. We propose deep negative correlation classification (DNCC) DNCC yields a deep classification ensemble where the individual estimator is both accurate and negatively correlated.
  arXiv  Detail & Related papers  (2022-12-14T07:35:20Z)
• Neural Architecture for Online Ensemble Continual Learning [6.241435193861262]
  We present a fully differentiable ensemble method that allows us to efficiently train an ensemble of neural networks in the end-to-end regime. The proposed technique achieves SOTA results without a memory buffer and clearly outperforms the reference methods.
  arXiv  Detail & Related papers  (2022-11-27T23:17:08Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)
• Learning with Neighbor Consistency for Noisy Labels [69.83857578836769]
  We present a method for learning from noisy labels that leverages similarities between training examples in feature space. We evaluate our method on datasets evaluating both synthetic (CIFAR-10, CIFAR-100) and realistic (mini-WebVision, Clothing1M, mini-ImageNet-Red) noise.
  arXiv  Detail & Related papers  (2022-02-04T15:46:27Z)
• Transductive Few-Shot Learning: Clustering is All You Need? [31.21306826132773]
  We investigate a general formulation for transive few-shot learning, which integrates prototype-based objectives. We find that our method yields competitive performances, in term of accuracy and optimization, while scaling up to large problems. Surprisingly, we find that our general model already achieve competitive performances in comparison to the state-of-the-art learning.
  arXiv  Detail & Related papers  (2021-06-16T16:14:01Z)
• No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data [78.69828864672978]
  A central challenge in training classification models in the real-world federated system is learning with non-IID data. We propose a novel and simple algorithm called Virtual Representations (CCVR), which adjusts the classifier using virtual representations sampled from an approximated ssian mixture model. Experimental results demonstrate that CCVR state-of-the-art performance on popular federated learning benchmarks including CIFAR-10, CIFAR-100, and CINIC-10.
  arXiv  Detail & Related papers  (2021-06-09T12:02:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.