Seeing the Forest for the Trees: The Gaussian Process Limit of BART

• URL: http://arxiv.org/abs/2607.28844v1
• Date: Thu, 30 Jul 2026 21:13:09 GMT
• Title: Seeing the Forest for the Trees: The Gaussian Process Limit of BART
• Abstract summary: Bayesian Additive Regression Trees (BART) have shown state-of-the-art performance in both prediction and causal inference problems.<n>Previous theoretical work has attempted to explain BART's superior performance by establishing posterior contraction rates for standard BART models.<n>We take a different approach and study the behavior of BART as the number of trees grows towards infinity.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Bayesian Additive Regression Trees (BART) have shown state-of-the-art performance in both prediction and causal inference problems. Previous theoretical work has attempted to explain BART's superior performance by establishing posterior contraction rates for standard BART models, but these rates depend strongly on the number of covariates. Here, we take a different approach and study the behavior of BART as the number of trees grows towards infinity. We show that in this regime, BART converges to a Gaussian process (GP) with a particular kernel. The kernel and its corresponding reproducing kernel Hilbert space (RKHS) have favorable inferential properties that help explain BART's excellent performance. We introduce *random tree features* as an approximation to this limiting GP, and establish minimax-optimal learning rates for ridge regression on these random features that depend only logarithmically on dimension. In addition to providing insight into the empirical success of BART, random tree features offer a computational benefit over traditional MCMC estimation. The random-features approximation also allows practitioners to easily incorporate BART into any model which has a linear predictor, expanding the applicability and flexibility of BART.

Related papers

• Probit Monotone BART [0.0]
  We propose probit monotone BART, which allows the monotone BART framework to estimate conditional mean functions when the outcome variable is binary.<n>BART of Chipman et al. (2010) has proven to be a powerful tool for nonparametric modeling and prediction.
  arXiv  Detail & Related papers  (2025-08-29T22:19:02Z)
• On the Gaussian process limit of Bayesian Additive Regression Trees [0.0]
  In the limit of infinite trees, BART becomes equivalent to Gaussian process (GP) regression.<n>This study opens new ways to understand and develop BART and GP regression.<n>The implementation of BART as GP is available in the Python package lsqfitgp.
  arXiv  Detail & Related papers  (2024-10-26T23:18:33Z)
• On the Sublinear Regret of GP-UCB [58.25014663727544]
  We show that the Gaussian Process Upper Confidence Bound (GP-UCB) algorithm enjoys nearly optimal regret rates. Our improvements rely on a key technical contribution -- regularizing kernel ridge estimators in proportion to the smoothness of the underlying kernel.
  arXiv  Detail & Related papers  (2023-07-14T13:56:11Z)
• Graph Signal Sampling for Inductive One-Bit Matrix Completion: a Closed-form Solution [112.3443939502313]
  We propose a unified graph signal sampling framework which enjoys the benefits of graph signal analysis and processing. The key idea is to transform each user's ratings on the items to a function (signal) on the vertices of an item-item graph. For the online setting, we develop a Bayesian extension, i.e., BGS-IMC which considers continuous random Gaussian noise in the graph Fourier domain.
  arXiv  Detail & Related papers  (2023-02-08T08:17:43Z)
• A Mixing Time Lower Bound for a Simplified Version of BART [5.149859291357858]
  We provide the first lower bound on the mixing time for a simplified version of BART. Inspired by this new connection between the mixing time and the number of data points, we perform rigorous simulations on BART. We show qualitatively that BART's mixing time increases with the number of data points.
  arXiv  Detail & Related papers  (2022-10-17T18:45:36Z)
• GP-BART: a novel Bayesian additive regression trees approach using Gaussian processes [1.03590082373586]
  The GP-BART model is an extension of BART which addresses the limitation by assuming GP priors for the predictions of each terminal node among all trees. The model's effectiveness is demonstrated through applications to simulated and real-world data, surpassing the performance of traditional modeling approaches in various scenarios.
  arXiv  Detail & Related papers  (2022-04-05T11:18:44Z)
• Optimal policy evaluation using kernel-based temporal difference methods [78.83926562536791]
  We use kernel Hilbert spaces for estimating the value function of an infinite-horizon discounted Markov reward process. We derive a non-asymptotic upper bound on the error with explicit dependence on the eigenvalues of the associated kernel operator. We prove minimax lower bounds over sub-classes of MRPs.
  arXiv  Detail & Related papers  (2021-09-24T14:48:20Z)
• Bayesian Bellman Operators [55.959376449737405]
  We introduce a novel perspective on Bayesian reinforcement learning (RL) Our framework is motivated by the insight that when bootstrapping is introduced, model-free approaches actually infer a posterior over Bellman operators, not value functions.
  arXiv  Detail & Related papers  (2021-06-09T12:20:46Z)
• KG-BART: Knowledge Graph-Augmented BART for Generative Commonsense Reasoning [78.81080813406177]
  We propose a novel knowledge graph augmented pre-trained language generation model KG-BART. KG-BART encompasses the complex relations of concepts through the knowledge graph and produces more logical and natural sentences as output.
  arXiv  Detail & Related papers  (2020-09-26T19:57:49Z)
• Bayesian Additive Regression Trees with Model Trees [0.0]
  We introduce an extension of BART, called Model Trees BART (MOTR-BART) MOTR-BART considers piecewise linear functions at node levels instead of piecewise constants. In our approach, local linearities are captured more efficiently and fewer trees are required to achieve equal or better performance than BART.
  arXiv  Detail & Related papers  (2020-06-12T22:19:58Z)
• Learning Near Optimal Policies with Low Inherent Bellman Error [115.16037976819331]
  We study the exploration problem with approximate linear action-value functions in episodic reinforcement learning. We show that exploration is possible using only emphbatch assumptions with an algorithm that achieves the optimal statistical rate for the setting we consider.
  arXiv  Detail & Related papers  (2020-02-29T02:02:40Z)
• Near-linear Time Gaussian Process Optimization with Adaptive Batching and Resparsification [119.41129787351092]
  We introduce BBKB, the first no-regret GP optimization algorithm that provably runs in near-linear time and selects candidates in batches. We show that the same bound can be used to adaptively delay costly updates to the sparse GP approximation, achieving a near-constant per-step amortized cost.
  arXiv  Detail & Related papers  (2020-02-23T17:43:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.