SoftBart: Soft Bayesian Additive Regression Trees

• URL: http://arxiv.org/abs/2210.16375v1
• Date: Fri, 28 Oct 2022 19:25:45 GMT
• Title: SoftBart: Soft Bayesian Additive Regression Trees
• Authors: Antonio R. Linero
• Abstract summary: This paper introduces the SoftBart package for fitting the Soft BART algorithm of Linero and Yang. A major goal of this package has been to facilitate the inclusion of BART in larger models. I show both how to use this package for standard prediction tasks and how to embed BART models in larger models.
• Score: 2.969705152497174
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Bayesian additive regression tree (BART) models have seen increased attention in recent years as a general-purpose nonparametric modeling technique. BART combines the flexibility of modern machine learning techniques with the principled uncertainty quantification of Bayesian inference, and it has been shown to be uniquely appropriate for addressing the high-noise problems that occur commonly in many areas of science, including medicine and the social sciences. This paper introduces the SoftBart package for fitting the Soft BART algorithm of Linero and Yang (2018). In addition to improving upon the predictive performance of other BART packages, a major goal of this package has been to facilitate the inclusion of BART in larger models, making it ideal for researchers in Bayesian statistics. I show both how to use this package for standard prediction tasks and how to embed BART models in larger models; I illustrate by using SoftBart to implement a nonparametric probit regression model, a semiparametric varying coefficient model, and a partial linear model.

Related papers

• Margin Matching Preference Optimization: Enhanced Model Alignment with Granular Feedback [64.67540769692074]
  Large language models (LLMs) fine-tuned with alignment techniques, such as reinforcement learning from human feedback, have been instrumental in developing some of the most capable AI systems to date. We introduce an approach called Margin Matching Preference Optimization (MMPO), which incorporates relative quality margins into optimization, leading to improved LLM policies and reward models. Experiments with both human and AI feedback data demonstrate that MMPO consistently outperforms baseline methods, often by a substantial margin, on popular benchmarks including MT-bench and RewardBench.
  arXiv  Detail & Related papers  (2024-10-04T04:56:11Z)
• ASBART:Accelerated Soft Bayes Additive Regression Trees [8.476756500467689]
  Soft BART improves both practically and heoretically on existing Bayesian sum-of-trees models. Compared to BART,it use more than about 20 times to complete the calculation with the default setting. We proposed a variant of BART named accelerate Soft BART(ASBART)
  arXiv  Detail & Related papers  (2023-10-21T11:27:42Z)
• Quasi Black-Box Variational Inference with Natural Gradients for Bayesian Learning [84.90242084523565]
  We develop an optimization algorithm suitable for Bayesian learning in complex models. Our approach relies on natural gradient updates within a general black-box framework for efficient training with limited model-specific derivations.
  arXiv  Detail & Related papers  (2022-05-23T18:54:27Z)
• Hierarchical Embedded Bayesian Additive Regression Trees [0.0]
  HE-BART allows for random effects to be included at the terminal node level of a set of regression trees. Using simulated and real-world examples, we demonstrate that HE-BART yields superior predictions for many of the standard mixed effects models' example data sets. In a future version of this paper, we outline its use in larger, more advanced data sets and structures.
  arXiv  Detail & Related papers  (2022-04-14T19:56:03Z)
• GP-BART: a novel Bayesian additive regression trees approach using Gaussian processes [1.03590082373586]
  The GP-BART model is an extension of BART which addresses the limitation by assuming GP priors for the predictions of each terminal node among all trees. The model's effectiveness is demonstrated through applications to simulated and real-world data, surpassing the performance of traditional modeling approaches in various scenarios.
  arXiv  Detail & Related papers  (2022-04-05T11:18:44Z)
• Generalized Bayesian Additive Regression Trees Models: Beyond Conditional Conjugacy [2.969705152497174]
  In this article, we greatly expand the domain of applicability of BART to arbitrary emphgeneralized BART models. Our algorithm requires only that the user be able to compute the likelihood and (optionally) its gradient and Fisher information. The potential applications are very broad; we consider examples in survival analysis, structured heteroskedastic regression, and gamma shape regression.
  arXiv  Detail & Related papers  (2022-02-20T22:52:07Z)
• Accounting for shared covariates in semi-parametric Bayesian additive regression trees [0.0]
  We propose some extensions to semi-parametric models based on Bayesian additive regression trees (BART) The main novelty in our approach lies in the way we change the tree-generation moves in BART to deal with this bias. We show competitive performance when compared to regression models, alternative formulations of semi-parametric BART, and other tree-based methods.
  arXiv  Detail & Related papers  (2021-08-17T13:58:44Z)
• Nonparametric Estimation in the Dynamic Bradley-Terry Model [69.70604365861121]
  We develop a novel estimator that relies on kernel smoothing to pre-process the pairwise comparisons over time. We derive time-varying oracle bounds for both the estimation error and the excess risk in the model-agnostic setting.
  arXiv  Detail & Related papers  (2020-02-28T21:52:49Z)
• Learnable Bernoulli Dropout for Bayesian Deep Learning [53.79615543862426]
  Learnable Bernoulli dropout (LBD) is a new model-agnostic dropout scheme that considers the dropout rates as parameters jointly optimized with other model parameters. LBD leads to improved accuracy and uncertainty estimates in image classification and semantic segmentation.
  arXiv  Detail & Related papers  (2020-02-12T18:57:14Z)
• Particle-Gibbs Sampling For Bayesian Feature Allocation Models [77.57285768500225]
  Most widely used MCMC strategies rely on an element wise Gibbs update of the feature allocation matrix. We have developed a Gibbs sampler that can update an entire row of the feature allocation matrix in a single move. This sampler is impractical for models with a large number of features as the computational complexity scales exponentially in the number of features. We develop a Particle Gibbs sampler that targets the same distribution as the row wise Gibbs updates, but has computational complexity that only grows linearly in the number of features.
  arXiv  Detail & Related papers  (2020-01-25T22:11:51Z)
• AvgOut: A Simple Output-Probability Measure to Eliminate Dull Responses [97.50616524350123]
  We build dialogue models that are dynamically aware of what utterances or tokens are dull without any feature-engineering. The first model, MinAvgOut, directly maximizes the diversity score through the output distributions of each batch. The second model, Label Fine-Tuning (LFT), prepends to the source sequence a label continuously scaled by the diversity score to control the diversity level. The third model, RL, adopts Reinforcement Learning and treats the diversity score as a reward signal.
  arXiv  Detail & Related papers  (2020-01-15T18:32:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.