Accounting for shared covariates in semi-parametric Bayesian additive regression trees

• URL: http://arxiv.org/abs/2108.07636v7
• Date: Tue, 30 Jul 2024 14:40:07 GMT
• Title: Accounting for shared covariates in semi-parametric Bayesian additive regression trees
• Authors: Estevão B. Prado, Andrew C. Parnell, Keefe Murphy, Nathan McJames, Ann O'Shea, Rafael A. Moral,
• Abstract summary: We propose some extensions to semi-parametric models based on Bayesian additive regression trees (BART) The main novelty in our approach lies in the way we change the tree-generation moves in BART to deal with this bias. We show competitive performance when compared to regression models, alternative formulations of semi-parametric BART, and other tree-based methods.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose some extensions to semi-parametric models based on Bayesian additive regression trees (BART). In the semi-parametric BART paradigm, the response variable is approximated by a linear predictor and a BART model, where the linear component is responsible for estimating the main effects and BART accounts for non-specified interactions and non-linearities. Previous semi-parametric models based on BART have assumed that the set of covariates in the linear predictor and the BART model are mutually exclusive in an attempt to avoid poor coverage properties and reduce bias in the estimates of the parameters in the linear predictor. The main novelty in our approach lies in the way we change the tree-generation moves in BART to deal with this bias and resolve non-identifiability issues between the parametric and non-parametric components, even when they have covariates in common. This allows us to model complex interactions involving the covariates of primary interest, both among themselves and with those in the BART component. Our novel method is developed with a view to analysing data from an international education assessment, where certain predictors of students' achievements in mathematics are of particular interpretational interest. Through additional simulation studies and another application to a well-known benchmark dataset, we also show competitive performance when compared to regression models, alternative formulations of semi-parametric BART, and other tree-based methods. The implementation of the proposed method is available at \url{https://github.com/ebprado/CSP-BART}.

Related papers

• Linear-cost unbiased posterior estimates for crossed effects and matrix factorization models via couplings [0.0]
  We design and analyze unbiased Markov chain Monte Carlo schemes based on couplings of blocked Gibbs samplers (BGSs) Our methodology is designed for and applicable to high-dimensional BGS with conditionally independent blocks.
  arXiv  Detail & Related papers  (2024-10-11T16:05:01Z)
• Multivariate root-n-consistent smoothing parameter free matching estimators and estimators of inverse density weighted expectations [51.000851088730684]
  We develop novel modifications of nearest-neighbor and matching estimators which converge at the parametric $sqrt n $-rate. We stress that our estimators do not involve nonparametric function estimators and in particular do not rely on sample-size dependent parameters smoothing.
  arXiv  Detail & Related papers  (2024-07-11T13:28:34Z)
• Co-data Learning for Bayesian Additive Regression Trees [0.0]
  We propose to incorporate co-data into a sum-of-trees prediction model. The proposed method can handle multiple types of co-data simultaneously. Co-data enhances prediction in an application to diffuse large B-cell lymphoma prognosis.
  arXiv  Detail & Related papers  (2023-11-16T16:14:39Z)
• SoftBart: Soft Bayesian Additive Regression Trees [2.969705152497174]
  This paper introduces the SoftBart package for fitting the Soft BART algorithm of Linero and Yang. A major goal of this package has been to facilitate the inclusion of BART in larger models. I show both how to use this package for standard prediction tasks and how to embed BART models in larger models.
  arXiv  Detail & Related papers  (2022-10-28T19:25:45Z)
• Hierarchical Embedded Bayesian Additive Regression Trees [0.0]
  HE-BART allows for random effects to be included at the terminal node level of a set of regression trees. Using simulated and real-world examples, we demonstrate that HE-BART yields superior predictions for many of the standard mixed effects models' example data sets. In a future version of this paper, we outline its use in larger, more advanced data sets and structures.
  arXiv  Detail & Related papers  (2022-04-14T19:56:03Z)
• GP-BART: a novel Bayesian additive regression trees approach using Gaussian processes [1.03590082373586]
  The GP-BART model is an extension of BART which addresses the limitation by assuming GP priors for the predictions of each terminal node among all trees. The model's effectiveness is demonstrated through applications to simulated and real-world data, surpassing the performance of traditional modeling approaches in various scenarios.
  arXiv  Detail & Related papers  (2022-04-05T11:18:44Z)
• Learning to Estimate Without Bias [57.82628598276623]
  Gauss theorem states that the weighted least squares estimator is a linear minimum variance unbiased estimation (MVUE) in linear models. In this paper, we take a first step towards extending this result to non linear settings via deep learning with bias constraints. A second motivation to BCE is in applications where multiple estimates of the same unknown are averaged for improved performance.
  arXiv  Detail & Related papers  (2021-10-24T10:23:51Z)
• Evaluating Sensitivity to the Stick-Breaking Prior in Bayesian Nonparametrics [85.31247588089686]
  We show that variational Bayesian methods can yield sensitivities with respect to parametric and nonparametric aspects of Bayesian models. We provide both theoretical and empirical support for our variational approach to Bayesian sensitivity analysis.
  arXiv  Detail & Related papers  (2021-07-08T03:40:18Z)
• A Hypergradient Approach to Robust Regression without Correspondence [85.49775273716503]
  We consider a variant of regression problem, where the correspondence between input and output data is not available. Most existing methods are only applicable when the sample size is small. We propose a new computational framework -- ROBOT -- for the shuffled regression problem.
  arXiv  Detail & Related papers  (2020-11-30T21:47:38Z)
• Identification of Probability weighted ARX models with arbitrary domains [75.91002178647165]
  PieceWise Affine models guarantees universal approximation, local linearity and equivalence to other classes of hybrid system. In this work, we focus on the identification of PieceWise Auto Regressive with eXogenous input models with arbitrary regions (NPWARX) The architecture is conceived following the Mixture of Expert concept, developed within the machine learning field.
  arXiv  Detail & Related papers  (2020-09-29T12:50:33Z)
• Machine learning for causal inference: on the use of cross-fit estimators [77.34726150561087]
  Doubly-robust cross-fit estimators have been proposed to yield better statistical properties. We conducted a simulation study to assess the performance of several estimators for the average causal effect (ACE) When used with machine learning, the doubly-robust cross-fit estimators substantially outperformed all of the other estimators in terms of bias, variance, and confidence interval coverage.
  arXiv  Detail & Related papers  (2020-04-21T23:09:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.