OGBoost: A Python Package for Ordinal Gradient Boosting

• URL: http://arxiv.org/abs/2502.13456v1
• Date: Wed, 19 Feb 2025 06:06:12 GMT
• Title: OGBoost: A Python Package for Ordinal Gradient Boosting
• Authors: Mansour T. A. Sharabiani, Alex Bottle, Alireza S. Mahani,
• Abstract summary: We introduce OGBoost, a scikit-learn-compatible Python package for ordinal regression using gradient boosting. The package is available on PyPI and can be installed via "pip install ogboost"
• Score: 0.0
• License:
• Abstract: This paper introduces OGBoost, a scikit-learn-compatible Python package for ordinal regression using gradient boosting. Ordinal variables (e.g., rating scales, quality assessments) lie between nominal and continuous data, necessitating specialized methods that reflect their inherent ordering. Built on a coordinate-descent approach for optimization and the latent-variable framework for ordinal regression, OGBoost performs joint optimization of a latent continuous regression function (functional gradient descent) and a threshold vector that converts the latent continuous value into discrete class probabilities (classical gradient descent). In addition to the stanadard methods for scikit-learn classifiers, the GradientBoostingOrdinal class implements a "decision_function" that returns the (scalar) value of the latent function for each observation, which can be used as a high-resolution alternative to class labels for comparing and ranking observations. The class has the option to use cross-validation for early stopping rather than a single holdout validation set, a more robust approach for small and/or imbalanced datasets. Furthermore, users can select base learners with different underlying algorithms and/or hyperparameters for use throughout the boosting iterations, resulting in a `heterogeneous' ensemble approach that can be used as a more efficient alternative to hyperparameter tuning (e.g. via grid search). We illustrate the capabilities of OGBoost through examples, using the wine quality dataset from the UCI respository. The package is available on PyPI and can be installed via "pip install ogboost".

Related papers

• A Nearly Optimal Single Loop Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness [15.656614304616006]
  This paper studies the problem of bilevel optimization where the upper-level function is nonstationary with potentially unbounded smoothness and the lower-level function is convex. Existing algorithm relies on a nested loop, which crucially requires significant tuning efforts and is not practical.
  arXiv  Detail & Related papers  (2024-12-28T04:40:27Z)
• ALoRE: Efficient Visual Adaptation via Aggregating Low Rank Experts [71.91042186338163]
  ALoRE is a novel PETL method that reuses the hypercomplex parameterized space constructed by Kronecker product to Aggregate Low Rank Experts. Thanks to the artful design, ALoRE maintains negligible extra parameters and can be effortlessly merged into the frozen backbone.
  arXiv  Detail & Related papers  (2024-12-11T12:31:30Z)
• Gradient-based optimization for variational empirical Bayes multiple regression [2.6763498831034043]
  We propose alternative optimization approaches based on gradient-based (quasi-Newton) methods. We show that GradVI produces similar predictive performance and converges in fewer iterations when predictors are highly correlated. Our methods are implemented in an open-source Python software, GradVI.
  arXiv  Detail & Related papers  (2024-11-21T20:35:44Z)
• ELRA: Exponential learning rate adaption gradient descent optimization method [83.88591755871734]
  We present a novel, fast (exponential rate), ab initio (hyper-free) gradient based adaption. The main idea of the method is to adapt the $alpha by situational awareness. It can be applied to problems of any dimensions n and scales only linearly.
  arXiv  Detail & Related papers  (2023-09-12T14:36:13Z)
• Benchmarking state-of-the-art gradient boosting algorithms for classification [0.0]
  This work explores the use of gradient boosting in the context of classification. Four popular implementations, including original GBM algorithm and selected state-of-the-art gradient boosting frameworks, have been compared. An attempt was made to indicate a gradient boosting variant showing the right balance between effectiveness, reliability and ease of use.
  arXiv  Detail & Related papers  (2023-05-26T17:06:15Z)
• Gradient Boosted Binary Histogram Ensemble for Large-scale Regression [60.16351608335641]
  We propose a gradient boosting algorithm for large-scale regression problems called textitGradient Boosted Binary Histogram Ensemble (GBBHE) based on binary histogram partition and ensemble learning. In the experiments, compared with other state-of-the-art algorithms such as gradient boosted regression tree (GBRT), our GBBHE algorithm shows promising performance with less running time on large-scale datasets.
  arXiv  Detail & Related papers  (2021-06-03T17:05:40Z)
• Why Approximate Matrix Square Root Outperforms Accurate SVD in Global Covariance Pooling? [59.820507600960745]
  We propose a new GCP meta-layer that uses SVD in the forward pass, and Pad'e Approximants in the backward propagation to compute the gradients. The proposed meta-layer has been integrated into different CNN models and achieves state-of-the-art performances on both large-scale and fine-grained datasets.
  arXiv  Detail & Related papers  (2021-05-06T08:03:45Z)
• Exploiting Adam-like Optimization Algorithms to Improve the Performance of Convolutional Neural Networks [82.61182037130405]
  gradient descent (SGD) is the main approach for training deep networks. In this work, we compare Adam based variants based on the difference between the present and the past gradients. We have tested ensemble of networks and the fusion with ResNet50 trained with gradient descent.
  arXiv  Detail & Related papers  (2021-03-26T18:55:08Z)
• Classification and Feature Transformation with Fuzzy Cognitive Maps [0.3299672391663526]
  Fuzzy Cognitive Maps (FCMs) are considered a soft computing technique combining elements of fuzzy logic and recurrent neural networks. In this work we propose an FCM based classifier with a fully connected map structure. Weights were learned with a gradient algorithm and logloss or cross-entropy were used as the cost function.
  arXiv  Detail & Related papers  (2021-03-08T22:26:24Z)
• StochasticRank: Global Optimization of Scale-Free Discrete Functions [28.224889996383396]
  In this paper, we introduce a powerful and efficient framework for direct optimization of ranking metrics. We show that classic smoothing approaches may introduce bias and present a universal solution for a proper debiasing. Our framework applies to any scale-free discrete loss function.
  arXiv  Detail & Related papers  (2020-03-04T15:27:11Z)
• Variance Reduction with Sparse Gradients [82.41780420431205]
  Variance reduction methods such as SVRG and SpiderBoost use a mixture of large and small batch gradients. We introduce a new sparsity operator: The random-top-k operator. Our algorithm consistently outperforms SpiderBoost on various tasks including image classification, natural language processing, and sparse matrix factorization.
  arXiv  Detail & Related papers  (2020-01-27T08:23:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.