An Infinite BART model

• URL: http://arxiv.org/abs/2511.20087v1
• Date: Tue, 25 Nov 2025 09:01:47 GMT
• Title: An Infinite BART model
• Authors: Marco Battiston, Yu Luo,
• Abstract summary: We propose a generalization of the BART model that has two main features.<n>It automatically selects the number of decision trees using the given data.<n>Each data point can only use a selection of weak learners, instead of all of them.
• Score: 5.209680411062657
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Bayesian additive regression trees (BART) are popular Bayesian ensemble models used in regression and classification analysis. Under this modeling framework, the regression function is approximated by an ensemble of decision trees, interpreted as weak learners that capture different features of the data. In this work, we propose a generalization of the BART model that has two main features: first, it automatically selects the number of decision trees using the given data; second, the model allows clusters of observations to have different regression functions since each data point can only use a selection of weak learners, instead of all of them. This model generalization is accomplished by including a binary weight matrix in the conditional distribution of the response variable, which activates only a specific subset of decision trees for each observation. Such a matrix is endowed with an Indian Buffet process prior, and sampled within the MCMC sampler, together with the other BART parameters. We then compare the Infinite BART model with the classic one on simulated and real datasets. Specifically, we provide examples illustrating variable importance, partial dependence and causal estimation.

Related papers

• MLCBART: Multilabel Classification with Bayesian Additive Regression Trees [0.6117371161379209]
  Multilabel Classification deals with the simultaneous classification of multiple binary labels.<n>BART is a nonparametric and flexible model structure capable of uncovering complex relationships within the data.<n>Our adaptation, MLCBART, assumes that labels arise from thresholding an underlying numeric scale.
  arXiv  Detail & Related papers  (2026-01-13T20:17:45Z)
• Joint Models for Handling Non-Ignorable Missing Data using Bayesian Additive Regression Trees: Application to Leaf Photosynthetic Traits Data [0.0]
  Dealing with missing data poses significant challenges in predictive analysis.<n>In cases where the data are missing not at random, jointly modeling the data and missing data indicators is essential.<n>We propose two methods under a selection model framework for handling data with missingness.
  arXiv  Detail & Related papers  (2024-12-19T15:26:55Z)
• Transformers meet Stochastic Block Models: Attention with Data-Adaptive Sparsity and Cost [53.746169882193456]
  Recent works have proposed various sparse attention modules to overcome the quadratic cost of self-attention. We propose a model that resolves both problems by endowing each attention head with a mixed-membership Block Model. Our model outperforms previous efficient variants as well as the original Transformer with full attention.
  arXiv  Detail & Related papers  (2022-10-27T15:30:52Z)
• Learning from aggregated data with a maximum entropy model [73.63512438583375]
  We show how a new model, similar to a logistic regression, may be learned from aggregated data only by approximating the unobserved feature distribution with a maximum entropy hypothesis. We present empirical evidence on several public datasets that the model learned this way can achieve performances comparable to those of a logistic model trained with the full unaggregated data.
  arXiv  Detail & Related papers  (2022-10-05T09:17:27Z)
• Variational Inference for Bayesian Bridge Regression [0.0]
  We study the implementation of Automatic Differentiation Variational inference (ADVI) for Bayesian inference on regression models with bridge penalization. The bridge approach uses $ell_alpha$ norm, with $alpha in (0, +infty)$ to define a penalization on large values of the regression coefficients. We illustrate the approach on non-parametric regression models with B-splines, although the method works seamlessly for other choices of basis functions.
  arXiv  Detail & Related papers  (2022-05-19T12:29:09Z)
• Hierarchical Embedded Bayesian Additive Regression Trees [0.0]
  HE-BART allows for random effects to be included at the terminal node level of a set of regression trees. Using simulated and real-world examples, we demonstrate that HE-BART yields superior predictions for many of the standard mixed effects models' example data sets. In a future version of this paper, we outline its use in larger, more advanced data sets and structures.
  arXiv  Detail & Related papers  (2022-04-14T19:56:03Z)
• GP-BART: a novel Bayesian additive regression trees approach using Gaussian processes [1.03590082373586]
  The GP-BART model is an extension of BART which addresses the limitation by assuming GP priors for the predictions of each terminal node among all trees. The model's effectiveness is demonstrated through applications to simulated and real-world data, surpassing the performance of traditional modeling approaches in various scenarios.
  arXiv  Detail & Related papers  (2022-04-05T11:18:44Z)
• CARMS: Categorical-Antithetic-REINFORCE Multi-Sample Gradient Estimator [60.799183326613395]
  We propose an unbiased estimator for categorical random variables based on multiple mutually negatively correlated (jointly antithetic) samples. CARMS combines REINFORCE with copula based sampling to avoid duplicate samples and reduce its variance, while keeping the estimator unbiased using importance sampling. We evaluate CARMS on several benchmark datasets on a generative modeling task, as well as a structured output prediction task, and find it to outperform competing methods including a strong self-control baseline.
  arXiv  Detail & Related papers  (2021-10-26T20:14:30Z)
• Flexible Model Aggregation for Quantile Regression [92.63075261170302]
  Quantile regression is a fundamental problem in statistical learning motivated by a need to quantify uncertainty in predictions. We investigate methods for aggregating any number of conditional quantile models. All of the models we consider in this paper can be fit using modern deep learning toolkits.
  arXiv  Detail & Related papers  (2021-02-26T23:21:16Z)
• Robust Finite Mixture Regression for Heterogeneous Targets [70.19798470463378]
  We propose an FMR model that finds sample clusters and jointly models multiple incomplete mixed-type targets simultaneously. We provide non-asymptotic oracle performance bounds for our model under a high-dimensional learning framework. The results show that our model can achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-10-12T03:27:07Z)
• Particle-Gibbs Sampling For Bayesian Feature Allocation Models [77.57285768500225]
  Most widely used MCMC strategies rely on an element wise Gibbs update of the feature allocation matrix. We have developed a Gibbs sampler that can update an entire row of the feature allocation matrix in a single move. This sampler is impractical for models with a large number of features as the computational complexity scales exponentially in the number of features. We develop a Particle Gibbs sampler that targets the same distribution as the row wise Gibbs updates, but has computational complexity that only grows linearly in the number of features.
  arXiv  Detail & Related papers  (2020-01-25T22:11:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.