A Leakage-Free Stacked Ensemble Method for Multiclass Classification

• URL: http://arxiv.org/abs/2607.22081v1
• Date: Fri, 24 Jul 2026 08:26:40 GMT
• Title: A Leakage-Free Stacked Ensemble Method for Multiclass Classification
• Abstract summary: We propose a Leakage-Free Stacked ensemble framework that integrates functional learning and rule-based learning for robust multiclass classification.<n>The framework constructs unbiased meta-features by employing a strict out-of-fold stacking strategy to ensure complete isolation between training and validation data.<n> Experimental evaluations on multi-class datasets demonstrate that LFS-FRAME improves performance metrics.
• Score: 1.1090533263502935
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multiclass classification is a fundamental problem across a wide range of domains. It is still challenging due to possession of high inter-class similarity, class imbalance datasets, and variability in data distributions. Rule-based classifiers such as XGBoost often achieve stronger performance on structured features, but they are limited in capturing smooth functional relationships among variables. Similarly, neural network models can represent complex nonlinear interactions but frequently suffer from overfitting and generalization issues. To address these limitations, we propose LFS-FRAME, a Leakage-Free Stacked ensemble framework that integrates functional learning using Kolmogorov-Arnold Networks (KAN) and rule-based learning via XGBoost for robust multiclass classification. The proposed framework constructs unbiased meta-features by employing a strict out-of-fold stacking strategy to ensure complete isolation between training and validation data hence preventing performance leakage. By learning over probabilistic outputs from heterogeneous base learners, the meta-classifier effectively exploits both global functional patterns and sharp decision boundaries present in the complex data. Experimental evaluations on multi-class datasets demonstrate that LFS-FRAME improves performance metrics, and overall accuracy is 89.85% in identifying major families and 81.74% in identifying sub-families relative to strong single-model baselines. These results highlight the effectiveness of leakage-free functional and rule-based stacking for reliable and generalizable multiclass classification.

Related papers

• FB-NLL: A Feature-Based Approach to Tackle Noisy Labels in Personalized Federated Learning [2.750124853532831]
  We propose FB-NLL, a feature-centric framework that decouples user clustering from iterative training dynamics.<n>Our framework consistently outperforms state-of-the-art baselines in terms of average accuracy and performance stability.
  arXiv  Detail & Related papers  (2026-04-21T17:51:58Z)
• i-IF-Learn: Iterative Feature Selection and Unsupervised Learning for High-Dimensional Complex Data [5.761834753104483]
  i-IF-Learn is an unsupervised framework that jointly performs feature selection and clustering.<n>Our core innovation is an adaptive feature selection statistic that effectively combines pseudo-label supervision with unsupervised signals.<n>Experiments on gene microarray and single-cell RNA-seq datasets show that i-IF-Learn significantly surpasses classical and deep clustering baselines.
  arXiv  Detail & Related papers  (2026-03-25T07:35:38Z)
• You Can Trust Your Clustering Model: A Parameter-free Self-Boosting Plug-in for Deep Clustering [73.48306836608124]
  DCBoost is a parameter-free plug-in designed to enhance the global feature structures of current deep clustering models.<n>By harnessing reliable local structural cues, our method aims to elevate clustering performance effectively.
  arXiv  Detail & Related papers  (2025-11-26T09:16:36Z)
• CKAA: Cross-subspace Knowledge Alignment and Aggregation for Robust Continual Learning [80.18781219542016]
  Continual Learning (CL) empowers AI models to continuously learn from sequential task streams.<n>Recent parameter-efficient fine-tuning (PEFT)-based CL methods have garnered increasing attention due to their superior performance.<n>We propose Cross-subspace Knowledge Alignment and Aggregation (CKAA) to enhance robustness against misleading task-ids.
  arXiv  Detail & Related papers  (2025-07-13T03:11:35Z)
• Intelligently Augmented Contrastive Tensor Factorization: Empowering Multi-dimensional Time Series Classification in Low-Data Environments [4.77513566805416]
  We propose a versatile yet data-efficient framework, Intelligently Augmented Contrastive Factorization (ITA-CTF)<n>ITA-CTF module learns effective representations from multi-dimensional time series.<n>It incorporates a new contrastive loss optimization to similarity learning and class-awareness.<n>Compared to standard and several DL benchmarks, notable performance improvements up to 18.7% were achieved.
  arXiv  Detail & Related papers  (2025-05-03T11:28:13Z)
• Long-Tailed Classification Based on Coarse-Grained Leading Forest and Multi-Center Loss [20.10399273585125]
  Long-tailed (LT) classification is an unavoidable and challenging problem in the real world. We propose a novel long-tailed classification framework, aiming to build a multi-granularity classification model by means of invariant feature learning. Our approach achieves state-of-the-art performance on both existing benchmarks ImageNet-GLT and MSCOCO-GLT.
  arXiv  Detail & Related papers  (2023-10-12T10:51:23Z)
• Extension of Transformational Machine Learning: Classification Problems [0.0]
  This study explores the application and performance of Transformational Machine Learning (TML) in drug discovery. TML, a meta learning algorithm, excels in exploiting common attributes across various domains. The drug discovery process, which is complex and time-consuming, can benefit greatly from the enhanced prediction accuracy.
  arXiv  Detail & Related papers  (2023-08-07T07:34:18Z)
• FaFCNN: A General Disease Classification Framework Based on Feature Fusion Neural Networks [4.097623533226476]
  We propose the Feature-aware Fusion Correlation Neural Network (FaFCNN), which introduces a feature-aware interaction module and a feature alignment module based on domain adversarial learning. The experimental results show that training using augmented features obtained by pre-training gradient boosting decision tree yields more performance gains than random-forest based methods.
  arXiv  Detail & Related papers  (2023-07-24T04:23:08Z)
• Deep Negative Correlation Classification [82.45045814842595]
  Existing deep ensemble methods naively train many different models and then aggregate their predictions. We propose deep negative correlation classification (DNCC) DNCC yields a deep classification ensemble where the individual estimator is both accurate and negatively correlated.
  arXiv  Detail & Related papers  (2022-12-14T07:35:20Z)
• Meta-Causal Feature Learning for Out-of-Distribution Generalization [71.38239243414091]
  This paper presents a balanced meta-causal learner (BMCL), which includes a balanced task generation module (BTG) and a meta-causal feature learning module (MCFL) BMCL effectively identifies the class-invariant visual regions for classification and may serve as a general framework to improve the performance of the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-08-22T09:07:02Z)
• Leveraging Ensembles and Self-Supervised Learning for Fully-Unsupervised Person Re-Identification and Text Authorship Attribution [77.85461690214551]
  Learning from fully-unlabeled data is challenging in Multimedia Forensics problems, such as Person Re-Identification and Text Authorship Attribution. Recent self-supervised learning methods have shown to be effective when dealing with fully-unlabeled data in cases where the underlying classes have significant semantic differences. We propose a strategy to tackle Person Re-Identification and Text Authorship Attribution by enabling learning from unlabeled data even when samples from different classes are not prominently diverse.
  arXiv  Detail & Related papers  (2022-02-07T13:08:11Z)
• Semi-supervised Domain Adaptive Structure Learning [72.01544419893628]
  Semi-supervised domain adaptation (SSDA) is a challenging problem requiring methods to overcome both 1) overfitting towards poorly annotated data and 2) distribution shift across domains. We introduce an adaptive structure learning method to regularize the cooperation of SSL and DA.
  arXiv  Detail & Related papers  (2021-12-12T06:11:16Z)
• Multi-Class Classification from Single-Class Data with Confidences [90.48669386745361]
  We propose an empirical risk minimization framework that is loss-/model-/optimizer-independent. We show that our method can be Bayes-consistent with a simple modification even if the provided confidences are highly noisy.
  arXiv  Detail & Related papers  (2021-06-16T15:38:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.