Semi-parametric Bayesian Additive Regression Trees
- URL: http://arxiv.org/abs/2108.07636v1
- Date: Tue, 17 Aug 2021 13:58:44 GMT
- Title: Semi-parametric Bayesian Additive Regression Trees
- Authors: Estev\~ao B. Prado, Andrew C. Parnell, Nathan McJames, Ann O'Shea,
Rafael A. Moral
- Abstract summary: We propose a new semi-parametric model based on Bayesian Additive Regression Trees (BART)
In our approach, the response variable is approximated by a linear predictor and a BART model, where the first component is responsible for estimating the main effects.
We demonstrate that the performance of the new semi-parametric BART is competitive when compared to regression models and other tree-based methods.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We propose a new semi-parametric model based on Bayesian Additive Regression
Trees (BART). In our approach, the response variable is approximated by a
linear predictor and a BART model, where the first component is responsible for
estimating the main effects and BART accounts for the non-specified
interactions and non-linearities. The novelty in our approach lies in the way
we change tree generation moves in BART to deal with confounding between the
parametric and non-parametric components when they have covariates in common.
Through synthetic and real-world examples, we demonstrate that the performance
of the new semi-parametric BART is competitive when compared to regression
models and other tree-based methods. The implementation of the proposed method
is available at https://github.com/ebprado/SP-BART.
Related papers
- REBAR: Retrieval-Based Reconstruction for Time-series Contrastive Learning [64.08293076551601]
We propose a novel method of using a learned measure for identifying positive pairs.
Our Retrieval-Based Reconstruction measure measures the similarity between two sequences.
We show that the REBAR error is a predictor of mutual class membership.
arXiv Detail & Related papers (2023-11-01T13:44:45Z) - Theory of Posterior Concentration for Generalized Bayesian Additive
Regression Trees [0.685316573653194]
We describe a Generalized framework for Bayesian trees and their additive ensembles.
We derive sufficient conditions on the response distribution, under which the posterior concentrates at a minimax rate, up to a logarithmic factor.
arXiv Detail & Related papers (2023-04-25T00:52:48Z) - SoftBart: Soft Bayesian Additive Regression Trees [2.969705152497174]
This paper introduces the SoftBart package for fitting the Soft BART algorithm of Linero and Yang.
A major goal of this package has been to facilitate the inclusion of BART in larger models.
I show both how to use this package for standard prediction tasks and how to embed BART models in larger models.
arXiv Detail & Related papers (2022-10-28T19:25:45Z) - Hierarchical Embedded Bayesian Additive Regression Trees [0.0]
HE-BART allows for random effects to be included at the terminal node level of a set of regression trees.
Using simulated and real-world examples, we demonstrate that HE-BART yields superior predictions for many of the standard mixed effects models' example data sets.
In a future version of this paper, we outline its use in larger, more advanced data sets and structures.
arXiv Detail & Related papers (2022-04-14T19:56:03Z) - GP-BART: a novel Bayesian additive regression trees approach using
Gaussian processes [1.03590082373586]
The GP-BART model is an extension of BART which addresses the limitation by assuming GP priors for the predictions of each terminal node among all trees.
The model's effectiveness is demonstrated through applications to simulated and real-world data, surpassing the performance of traditional modeling approaches in various scenarios.
arXiv Detail & Related papers (2022-04-05T11:18:44Z) - Momentum Pseudo-Labeling for Semi-Supervised Speech Recognition [55.362258027878966]
We present momentum pseudo-labeling (MPL) as a simple yet effective strategy for semi-supervised speech recognition.
MPL consists of a pair of online and offline models that interact and learn from each other, inspired by the mean teacher method.
The experimental results demonstrate that MPL effectively improves over the base model and is scalable to different semi-supervised scenarios.
arXiv Detail & Related papers (2021-06-16T16:24:55Z) - MINIMALIST: Mutual INformatIon Maximization for Amortized Likelihood
Inference from Sampled Trajectories [61.3299263929289]
Simulation-based inference enables learning the parameters of a model even when its likelihood cannot be computed in practice.
One class of methods uses data simulated with different parameters to infer an amortized estimator for the likelihood-to-evidence ratio.
We show that this approach can be formulated in terms of mutual information between model parameters and simulated data.
arXiv Detail & Related papers (2021-06-03T12:59:16Z) - GELATO: Geometrically Enriched Latent Model for Offline Reinforcement
Learning [54.291331971813364]
offline reinforcement learning approaches can be divided into proximal and uncertainty-aware methods.
In this work, we demonstrate the benefit of combining the two in a latent variational model.
Our proposed metrics measure both the quality of out of distribution samples as well as the discrepancy of examples in the data.
arXiv Detail & Related papers (2021-02-22T19:42:40Z) - Online nonparametric regression with Sobolev kernels [99.12817345416846]
We derive the regret upper bounds on the classes of Sobolev spaces $W_pbeta(mathcalX)$, $pgeq 2, beta>fracdp$.
The upper bounds are supported by the minimax regret analysis, which reveals that in the cases $beta> fracd2$ or $p=infty$ these rates are (essentially) optimal.
arXiv Detail & Related papers (2021-02-06T15:05:14Z) - Probabilistic Case-based Reasoning for Open-World Knowledge Graph
Completion [59.549664231655726]
A case-based reasoning (CBR) system solves a new problem by retrieving cases' that are similar to the given problem.
In this paper, we demonstrate that such a system is achievable for reasoning in knowledge-bases (KBs)
Our approach predicts attributes for an entity by gathering reasoning paths from similar entities in the KB.
arXiv Detail & Related papers (2020-10-07T17:48:12Z) - Bayesian Additive Regression Trees with Model Trees [0.0]
We introduce an extension of BART, called Model Trees BART (MOTR-BART)
MOTR-BART considers piecewise linear functions at node levels instead of piecewise constants.
In our approach, local linearities are captured more efficiently and fewer trees are required to achieve equal or better performance than BART.
arXiv Detail & Related papers (2020-06-12T22:19:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.