MorphBoost: Self-Organizing Universal Gradient Boosting with Adaptive Tree Morphing

• URL: http://arxiv.org/abs/2511.13234v1
• Date: Mon, 17 Nov 2025 10:54:01 GMT
• Title: MorphBoost: Self-Organizing Universal Gradient Boosting with Adaptive Tree Morphing
• Authors: Boris Kriuk,
• Abstract summary: MorphBoost is a new gradient boosting framework featuring self-organizing tree structures that dynamically morph their splitting behavior during training.<n>The algorithm implements adaptive split functions that evolve based on accumulated gradient statistics and robustness-dependent learning pressures.<n>MorphBoost secured the overall winner position with 4/10 dataset wins (40% win rate) and 6/30 top-3 finishes (20%), while maintaining the lowest variance (948) and highest minimum accuracy across all models.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traditional gradient boosting algorithms employ static tree structures with fixed splitting criteria that remain unchanged throughout training, limiting their ability to adapt to evolving gradient distributions and problem-specific characteristics across different learning stages. This work introduces MorphBoost, a new gradient boosting framework featuring self-organizing tree structures that dynamically morph their splitting behavior during training. The algorithm implements adaptive split functions that evolve based on accumulated gradient statistics and iteration-dependent learning pressures, enabling automatic adjustment to problem complexity. Key innovations include: (1) morphing split criterion combining gradient-based scores with information-theoretic metrics weighted by training progress; (2) automatic problem fingerprinting for intelligent parameter configuration across binary/multiclass/regression tasks; (3) vectorized tree prediction achieving significant computational speedups; (4) interaction-aware feature importance detecting multiplicative relationships; and (5) fast-mode optimization balancing speed and accuracy. Comprehensive benchmarking across 10 diverse datasets against competitive models (XGBoost, LightGBM, GradientBoosting, HistGradientBoosting, ensemble methods) demonstrates that MorphBoost achieves state-of-the-art performance, outperforming XGBoost by 0.84% on average. MorphBoost secured the overall winner position with 4/10 dataset wins (40% win rate) and 6/30 top-3 finishes (20%), while maintaining the lowest variance (σ=0.0948) and highest minimum accuracy across all models, revealing superior consistency and robustness. Performance analysis across difficulty levels shows competitive results on easy datasets while achieving notable improvements on advanced problems due to higher adaptation levels.

Related papers

• Beyond Softmax: A Natural Parameterization for Categorical Random Variables [61.709831225296305]
  We introduce the $textitcatnat$ function, a function composed of a sequence of hierarchical binary splits.<n>A rich set of experiments show that the proposed function improves the learning efficiency and yields models characterized by consistently higher test performance.
  arXiv  Detail & Related papers  (2025-09-29T12:55:50Z)
• Task adaption by biologically inspired stochastic comodulation [8.59194778459436]
  We show that fine-tuning convolutional networks by gain modulation improves on deterministic gain modulation. Our results suggest that comodulation representations can enhance learning efficiency and performance in multi-task learning.
  arXiv  Detail & Related papers  (2023-11-25T15:21:03Z)
• ELRA: Exponential learning rate adaption gradient descent optimization method [83.88591755871734]
  We present a novel, fast (exponential rate), ab initio (hyper-free) gradient based adaption. The main idea of the method is to adapt the $alpha by situational awareness. It can be applied to problems of any dimensions n and scales only linearly.
  arXiv  Detail & Related papers  (2023-09-12T14:36:13Z)
• Break a Lag: Triple Exponential Moving Average for Enhanced Optimization [2.0199251985015434]
  We introduce Fast Adaptive Moment Estimation (FAME), a novel optimization technique that leverages the power of Triple Exponential Moving Average.<n>FAME enhances responsiveness to data dynamics, mitigates trend identification lag, and optimize learning efficiency.<n>Our comprehensive evaluation encompasses different computer vision tasks including image classification, object detection, and semantic segmentation, integrating FAME into 30 distinct architectures.
  arXiv  Detail & Related papers  (2023-06-02T10:29:33Z)
• Benchmarking state-of-the-art gradient boosting algorithms for classification [0.0]
  This work explores the use of gradient boosting in the context of classification. Four popular implementations, including original GBM algorithm and selected state-of-the-art gradient boosting frameworks, have been compared. An attempt was made to indicate a gradient boosting variant showing the right balance between effectiveness, reliability and ease of use.
  arXiv  Detail & Related papers  (2023-05-26T17:06:15Z)
• Performance Embeddings: A Similarity-based Approach to Automatic Performance Optimization [71.69092462147292]
  Performance embeddings enable knowledge transfer of performance tuning between applications. We demonstrate this transfer tuning approach on case studies in deep neural networks, dense and sparse linear algebra compositions, and numerical weather prediction stencils.
  arXiv  Detail & Related papers  (2023-03-14T15:51:35Z)
• A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks [65.34977803841007]
  Predictive coding networks are neuroscience-inspired models with roots in both Bayesian statistics and neuroscience. We show how by simply changing the temporal scheduling of the update rule for the synaptic weights leads to an algorithm that is much more efficient and stable than the original one.
  arXiv  Detail & Related papers  (2022-11-16T00:11:04Z)
• Efficient Adaptive Ensembling for Image Classification [3.7241274058257092]
  We propose a novel method to boost image classification performances without increasing complexity. We trained two EfficientNet-b0 end-to-end models on disjoint subsets of data. We were able to outperform the state-of-the-art by an average of 0.5$%$ on the accuracy.
  arXiv  Detail & Related papers  (2022-06-15T08:55:47Z)
• Joint inference and input optimization in equilibrium networks [68.63726855991052]
  deep equilibrium model is a class of models that foregoes traditional network depth and instead computes the output of a network by finding the fixed point of a single nonlinear layer. We show that there is a natural synergy between these two settings. We demonstrate this strategy on various tasks such as training generative models while optimizing over latent codes, training models for inverse problems like denoising and inpainting, adversarial training and gradient based meta-learning.
  arXiv  Detail & Related papers  (2021-11-25T19:59:33Z)
• agtboost: Adaptive and Automatic Gradient Tree Boosting Computations [0.0]
  agtboost implements fast gradient tree boosting computations. A useful model validation function performs the Kolmogorov-Smirnov test on the learned distribution.
  arXiv  Detail & Related papers  (2020-08-28T12:42:19Z)
• Soft Gradient Boosting Machine [72.54062017726154]
  We propose the soft Gradient Boosting Machine (sGBM) by wiring multiple differentiable base learners together. Experimental results showed that, sGBM enjoys much higher time efficiency with better accuracy, given the same base learner in both on-line and off-line settings.
  arXiv  Detail & Related papers  (2020-06-07T06:43:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.