Gradient boosting with vector-valued leafs

• URL: http://arxiv.org/abs/2606.29326v1
• Date: Sun, 28 Jun 2026 10:42:29 GMT
• Title: Gradient boosting with vector-valued leafs
• Abstract summary: Gradient boosting in the form of decision tree ensembles has successfully been applied to a variety of problems.<n>This work extends the usual gradient boosting framework to functions of vector inputs and sketches a simple algorithm that can be used efficiently with histogram-based decision trees.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Gradient boosting in the form of decision tree ensembles has successfully been applied to a variety of problems using simple objective functions based on log-likelihoods of a single variable. The concept extends naturally to objective functions operating on vectors - for example, multinomial logistic log-likelihood for multi-class classification, where observations have a score for each class - but popular frameworks approach these functions by either updating one value of the input vectors at a time, or by using a diagonal upper bound on the second derivative. This work extends the usual gradient boosting framework to functions of vector inputs and sketches a simple algorithm that can be used efficiently with histogram-based decision trees.

Related papers

• Path-Based Gradient Boosting for Graph-Level Prediction [0.6181093777643575]
  PathBoost learns discriminative path-based features directly from the input graph structure.<n>We compared PathBoost to graph neural networks and graph kernel approaches on several benchmark datasets.<n>Overall, the results demonstrate that path-based boosting methods can be competitive with more complex black-box approaches.
  arXiv  Detail & Related papers  (2026-04-26T07:59:33Z)
• Multiclass Graph-Based Large Margin Classifiers: Unified Approach for Support Vectors and Neural Networks [5.58422527196734]
  This article presents advances in the use of Gabriel graphs (GGs) in binary and multiclass classification problems.<n>We discuss how activation functions and support edge (SE)-centered neurons affect the classification, proposing smoother functions and structural SV (SSV)-centered neurons.<n>We extend the neural network architecture, which can be trained with backpropagation with a softmax function and a cross-entropy loss.
  arXiv  Detail & Related papers  (2025-12-15T15:00:13Z)
• Beyond Softmax: A Natural Parameterization for Categorical Random Variables [61.709831225296305]
  We introduce the $textitcatnat$ function, a function composed of a sequence of hierarchical binary splits.<n>A rich set of experiments show that the proposed function improves the learning efficiency and yields models characterized by consistently higher test performance.
  arXiv  Detail & Related papers  (2025-09-29T12:55:50Z)
• OGBoost: A Python Package for Ordinal Gradient Boosting [0.0]
  We introduce OGBoost, a scikit-learn-compatible Python package for ordinal regression using gradient boosting.<n>The package is available on PyPI and can be installed via "pip install ogboost"
  arXiv  Detail & Related papers  (2025-02-19T06:06:12Z)
• Segmentation-guided Layer-wise Image Vectorization with Gradient Fills [6.037332707968933]
  We propose a segmentation-guided vectorization framework to convert images into concise vector graphics with gradient fills. With the guidance of an embedded gradient-aware segmentation, our approach progressively appends gradient-filled B'ezier paths to the output.
  arXiv  Detail & Related papers  (2024-08-28T12:08:25Z)
• Knowledge Composition using Task Vectors with Learned Anisotropic Scaling [51.4661186662329]
  We introduce aTLAS, an algorithm that linearly combines parameter blocks with different learned coefficients, resulting in anisotropic scaling at the task vector level. We show that such linear combinations explicitly exploit the low intrinsicity of pre-trained models, with only a few coefficients being the learnable parameters. We demonstrate the effectiveness of our method in task arithmetic, few-shot recognition and test-time adaptation, with supervised or unsupervised objectives.
  arXiv  Detail & Related papers  (2024-07-03T07:54:08Z)
• Dynamic Perceiver for Efficient Visual Recognition [87.08210214417309]
  We propose Dynamic Perceiver (Dyn-Perceiver) to decouple the feature extraction procedure and the early classification task. A feature branch serves to extract image features, while a classification branch processes a latent code assigned for classification tasks. Early exits are placed exclusively within the classification branch, thus eliminating the need for linear separability in low-level features.
  arXiv  Detail & Related papers  (2023-06-20T03:00:22Z)
• A Unified Framework for Implicit Sinkhorn Differentiation [58.56866763433335]
  We propose an algorithm that obtains analytical gradients of a Sinkhorn layer via implicit differentiation. We show that it is computationally more efficient, particularly when resources like GPU memory are scarce.
  arXiv  Detail & Related papers  (2022-05-13T14:45:31Z)
• Measure Inducing Classification and Regression Trees for Functional Data [0.0]
  We propose a tree-based algorithm for classification and regression problems in the context of functional data analysis. This is achieved by learning a weighted functional $L2$ space by means of constrained convex optimization.
  arXiv  Detail & Related papers  (2020-10-30T18:49:53Z)
• Channel-Directed Gradients for Optimization of Convolutional Neural Networks [50.34913837546743]
  We introduce optimization methods for convolutional neural networks that can be used to improve existing gradient-based optimization in terms of generalization error. We show that defining the gradients along the output channel direction leads to a performance boost, while other directions can be detrimental.
  arXiv  Detail & Related papers  (2020-08-25T00:44:09Z)
• Variance Reduction with Sparse Gradients [82.41780420431205]
  Variance reduction methods such as SVRG and SpiderBoost use a mixture of large and small batch gradients. We introduce a new sparsity operator: The random-top-k operator. Our algorithm consistently outperforms SpiderBoost on various tasks including image classification, natural language processing, and sparse matrix factorization.
  arXiv  Detail & Related papers  (2020-01-27T08:23:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.