Survival regression with accelerated failure time model in XGBoost

• URL: http://arxiv.org/abs/2006.04920v3
• Date: Sat, 21 Aug 2021 05:14:02 GMT
• Title: Survival regression with accelerated failure time model in XGBoost
• Authors: Avinash Barnwal, Hyunsu Cho, Toby Dylan Hocking
• Abstract summary: Survival regression is used to estimate the relation between time-to-event and feature variables. XGBoost implements loss functions for learning accelerated failure time models.
• Score: 1.5469452301122177
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Survival regression is used to estimate the relation between time-to-event and feature variables, and is important in application domains such as medicine, marketing, risk management and sales management. Nonlinear tree based machine learning algorithms as implemented in libraries such as XGBoost, scikit-learn, LightGBM, and CatBoost are often more accurate in practice than linear models. However, existing state-of-the-art implementations of tree-based models have offered limited support for survival regression. In this work, we implement loss functions for learning accelerated failure time (AFT) models in XGBoost, to increase the support for survival modeling for different kinds of label censoring. We demonstrate with real and simulated experiments the effectiveness of AFT in XGBoost with respect to a number of baselines, in two respects: generalization performance and training speed. Furthermore, we take advantage of the support for NVIDIA GPUs in XGBoost to achieve substantial speedup over multi-core CPUs. To our knowledge, our work is the first implementation of AFT that utilizes the processing power of NVIDIA GPUs. Starting from the 1.2.0 release, the XGBoost package natively supports the AFT model. The addition of AFT in XGBoost has had significant impact in the open source community, and a few statistics packages now utilize the XGBoost AFT model.

Related papers

• Very fast Bayesian Additive Regression Trees on GPU [0.0]
  I present a GPU-enabled implementation of BART, faster by up to 200x relative to a single CPU core, making BART competitive in running time with XGBoost. This implementation is available in the Python package bartz.
  arXiv  Detail & Related papers  (2024-10-30T17:29:03Z)
• Optimizing Foundation Model Inference on a Many-tiny-core Open-source RISC-V Platform [13.326025546527784]
  We present the first end-to-end inference results of transformer models on an open-source many-tiny-core RISC-V platform. For encoder-only models, we demonstrate a speedup of up to 12.8x between the most optimized implementation and the baseline version. For decoder-only topologies, we achieve 16.1x speedup in the Non-Autoregressive (NAR) mode and up to 35.6x speedup in the Autoregressive (AR) mode.
  arXiv  Detail & Related papers  (2024-05-29T17:16:59Z)
• Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment [56.44025052765861]
  Large language models (LLMs) have revolutionized Natural Language Processing (NLP), but their size creates computational bottlenecks. We introduce a novel approach to create accurate, sparse foundational versions of performant LLMs. We show a total speedup on CPUs for sparse-quantized LLaMA models of up to 8.6x.
  arXiv  Detail & Related papers  (2024-05-06T16:03:32Z)
• NeuraChip: Accelerating GNN Computations with a Hash-based Decoupled Spatial Accelerator [3.926150707772004]
  We introduce NeuraChip, a novel GNN spatial accelerator based on Gustavson's algorithm. NeuraChip decouples the multiplication and addition computations in sparse matrix multiplication. We also present NeuraSim, an open-source, cycle-accurate, multi-threaded, modular simulator for comprehensive performance analysis.
  arXiv  Detail & Related papers  (2024-04-23T20:51:09Z)
• AI and Memory Wall [81.06494558184049]
  We show how memory bandwidth can become the dominant bottleneck for decoder models. We argue for a redesign in model architecture, training, and deployment strategies to overcome this memory limitation.
  arXiv  Detail & Related papers  (2024-03-21T04:31:59Z)
• JaxMARL: Multi-Agent RL Environments and Algorithms in JAX [105.343918678781]
  We present JaxMARL, the first open-source, Python-based library that combines GPU-enabled efficiency with support for a large number of commonly used MARL environments. Our experiments show that, in terms of wall clock time, our JAX-based training pipeline is around 14 times faster than existing approaches. We also introduce and benchmark SMAX, a JAX-based approximate reimplementation of the popular StarCraft Multi-Agent Challenge.
  arXiv  Detail & Related papers  (2023-11-16T18:58:43Z)
• XGen-7B Technical Report [138.71625147048377]
  XGen is a series of 7B parameter models on up to 8K sequence length for up to 1.5T tokens. We open-source our models for both research advancements and commercial applications.
  arXiv  Detail & Related papers  (2023-09-07T02:20:03Z)
• Quantile Extreme Gradient Boosting for Uncertainty Quantification [1.7685947618629572]
  Extreme Gradient Boosting (XGBoost) is one of the most popular machine learning (ML) methods. We propose enhancements to XGBoost whereby a modified quantile regression is used as the objective function to estimate uncertainty (QXGBoost) Our proposed method had comparable or better performance than the uncertainty estimates generated for regular and quantile light gradient boosting.
  arXiv  Detail & Related papers  (2023-04-23T19:46:19Z)
• FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU [89.2451963569343]
  FlexGen is a generation engine for running large language model (LLM) inference on a single commodity GPU. When running OPT-175B on a single 16GB GPU, FlexGen achieves significantly higher throughput compared to state-of-the-art offloading systems. On the HELM benchmark, FlexGen can benchmark a 30B model with a 16GB GPU on 7 representative sub-scenarios in 21 hours.
  arXiv  Detail & Related papers  (2023-03-13T05:19:28Z)
• Multi-Target XGBoostLSS Regression [91.3755431537592]
  We present an extension of XGBoostLSS that models multiple targets and their dependencies in a probabilistic regression setting. Our approach outperforms existing GBMs with respect to runtime and compares well in terms of accuracy.
  arXiv  Detail & Related papers  (2022-10-13T08:26:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.